From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 01:12:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5916D1065693 for ; Sun, 21 Feb 2010 01:12:59 +0000 (UTC) (envelope-from duane.hesser@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 01BBD8FC12 for ; Sun, 21 Feb 2010 01:12:58 +0000 (UTC) Received: by vws14 with SMTP id 14so611844vws.13 for ; Sat, 20 Feb 2010 17:12:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:message-id:date:from :to:subject:in-reply-to:references:reply-to:x-mailer:mime-version :content-type:content-transfer-encoding; bh=6fdtiH9vo6s2V+fQcylkSlQR0twC/8hmQeUSO4BcRho=; b=ctbsQoIUbZPZdU7N9xNkDkYn/J2HlsoT5PdDuwiZ6W3nmpKXK2TKln1tY/tyUahJnU OHW+brzecjo44J6nu+kSRInZcMxEvHEj9D5GbFUe3iTvC82jEmhs0vuI0DItMbt6aUen qyTlAXvAi+p7jTqnwxY7PFbw+37x4EcWxFUSk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; b=jognElsXdSNjcg2i+P2n//405XU3i1uOV2g+YaBYrNqk4YytAIone28xCVsWOISlCV pAGwrKSicbVDC3iT5s7Kxn/H21G7ai/kJND460oyxGcJgribPoc1jBrsMVOpRqzRTrak uZhzhqLlVW0Jy3IQypJ6+eLkrjWqFWk2QP5Dw= Received: by 10.220.127.4 with SMTP id e4mr5906749vcs.199.1266713213586; Sat, 20 Feb 2010 16:46:53 -0800 (PST) Received: from belinda.androcles.org ([64.146.146.203]) by mx.google.com with ESMTPS id 22sm16459480vws.14.2010.02.20.16.46.52 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 20 Feb 2010 16:46:52 -0800 (PST) Received: from belinda.androcles.org (localhost [127.0.0.1]) by belinda.androcles.org (8.14.2/8.14.2/8.13.3/1) with SMTP id o1L0kmDt026041 for ; Sat, 20 Feb 2010 16:46:48 -0800 (PST) (envelope-from duane.hesser@gmail.com) Message-Id: <201002210046.o1L0kmDt026041@belinda.androcles.org> Date: Sat, 20 Feb 2010 16:46:48 -0800 From: "Duane H. Hesser" To: freebsd-hackers@freebsd.org In-Reply-To: <201002200100.48161.jkim@FreeBSD.org> References: <20100217215940.GA19713@triton8.kn-bremen.de> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> <201002200100.48161.jkim@FreeBSD.org> X-Mailer: Sylpheed 2.5.0 (GTK+ 2.16.6; i386-portbld-freebsd6.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 21 Feb 2010 02:19:52 +0000 Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Duane.Hesser@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 01:12:59 -0000 Tim and Juergen I have a couple of suggestions which may help you with what you are trying to do. First, though, I can confirm that Unix character special tape drivers since at least V7 have always ignored seeks. They happily return the requested offset without feeling the need to actually *do* anything. The FreeBSD scsi_sa driver continues this tradition. Historically, all other Unix character special storage devices have been unable to respond to SEEK_END, presumably because they are either removable medium devices or hard disks which have movable "partitions", and no one ever bothered to arrange to query the medium for size. These devices do support seeking, and in fact it is readily possibly to seek past the end of the file, just as for regular files. Unix block special tape drivers have historically supported seeking. Somebody decided that FreeBSD doesn't need block devices and tossed them out; I think that's unfortunate. Last time I looked (long time ago) Linux had block device tapes, which did seek, and character special tape devices which did not. (Correct me if I'm wrong, please). I have a tar-compatible package of my own (tartools) which is about 20 years old but features the ability to fast_skip file data when listing or skipping unselected files on extract. It uses either seeking or mtio, and handles the TAPE vs non-TAPE character special problem by simply checking to see if the device responds to mtio commands. If it does, it's a tape, and can't seek (but can zip along using mtio). The last time I compiled it on Linux, the same situation applied, but that was about 8 years ago, and according to Jung-uk Kim the linux folks have finally done something about it. The 32 bit version of the ioctl does appear in the linux emulator in /usr/src/sys/compat/linux/linux_ioctl.c (I'm on freebsd 6.4/i386), and is implemented using an "fo_ioctl()" with argument DIOCGMEDIASIZE, which /usr/include/sys/disk.h defines as "_IOR('d', 129, u_int)". That appears to be a valid ioct arg, because several freebsd programs, including bsdlabel.c use ioctl(f, DIOCGMEDIASIZE, &mediasize) where mediasize is declared as off_t, which seems wrong on recent freebsds, where off_t is 8 bytes, but u_int In any case, an ioctl isn't going to be very portable at this point (although very handy). There's another way to get medium size if you need it. Tartools use medium size to enable multi-volume archives, and until recently required user input by option or "rc" file to specify volume size. I've been doing extensive upgrades and bug fixes on tartools the last 3 months, and I've recently added code which will now get the size of USB sticks, compact flash, etc. (even floppies) without requiring a command line argument. It is possible to seek way past the end of a character special file (and never know it). This isn't a problem for single file archives (although it might be risky) but if you want multiple volumes you really need a size (which may be less than the medium size). You can seek past the end of the medium, but you'll get an error if you try to read or write there, so it is possible to get an accurate size using a binary "seek and read" strategy to discover the highest sector number which is readable). This is getting a little long, so let me summarize the suggestions. 1. Disambiguate character special tape devices from other character special devices by issuing an mtio "status" ioctl (MTIOCGET) and checking the 'mt_type' byte in the status structure. If it is 0, the device is not a tape and will support seeking (presuming it is a storage device). It the type is non-zero then you can mark it non-seekable and either quit there or program the code to do the skipping via mtio. 2. If you need the size of a device partition or removable medium, use a binary search algorithm which seeks forward until a read at the offset fails, then cutting the offset in half, seeking and reading again, etc. It is usually possible to converge on a sector withing a couple of dozen seeks. On Sat, 20 Feb 2010 01:00:40 -0500 Jung-uk Kim wrote: > On Saturday 20 February 2010 12:20 am, Tim Kientzle wrote: > > Juergen, > > > > I was looking at your Linux code here and thought > > the technique of trying lseek(SEEK_END) might work. > > Unfortunately, it doesn't: lseek(fd, 0, SEEK_END) gives > > zero for both /dev/sa0 (a tape drive) and /dev/cd0 > > (an optical drive). Are you sure it works on Linux? > > Can you please try ioctl(fd, BLKGETSIZE64, &some_uint64_var) or > ioctl(fd, BLKGETSIZE, &some_u_long_var)? > > Jung-uk Kim > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > --------- Duane.Hesser@gmail.com From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 05:22:38 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D88411065670 for ; Sun, 21 Feb 2010 05:22:38 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 8D3E38FC12 for ; Sun, 21 Feb 2010 05:22:38 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o1L5Mjgr011034; Sun, 21 Feb 2010 05:22:46 GMT (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id y7gxiu66hsxbqdukma9kzd6evs; Sun, 21 Feb 2010 05:22:45 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4B80C359.6070707@freebsd.org> Date: Sat, 20 Feb 2010 21:23:37 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090601 SeaMonkey/1.1.16 MIME-Version: 1.0 To: Duane.Hesser@gmail.com References: <20100217215940.GA19713@triton8.kn-bremen.de> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> <201002200100.48161.jkim@FreeBSD.org> <201002210046.o1L0kmDt026041@belinda.androcles.org> In-Reply-To: <201002210046.o1L0kmDt026041@belinda.androcles.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 05:22:38 -0000 Duane H. Hesser wrote: > > I have a couple of suggestions which may help you with what you are > trying to do. > > First, though, I can confirm that Unix character special tape > drivers since at least V7 have always ignored seeks. They happily > return the requested offset without feeling the need to actually > *do* anything. The FreeBSD scsi_sa driver continues this tradition. Duane, Thank you very, very much for taking the time to write up this information. I'm going to definitely archive your message for future reference. To clarify "what we are trying to do": For some time now, libarchive format handlers have issued "skip" requests internally for things like skipping unnecessary file bodies. Depending on the device you're talking to, these internal skip requests can be translated into seek operations or can be handled by reading and discarding data. As Juergen found when he tried to do "tar tf /dev/cd0", my earlier code was pretty conservative: It only translated skip requests into lseek() operations for regular files. Juergen took a stab at improving that so that operations on raw disk devices would use lseek(). With your information, it should be pretty easy to translate skip requests into mtio operations on tape drives. Libarchive's core already supports rounding skip operations to multiples of the block size, so this should be an easy addition. As it happens, I'm just starting this week to look into multi-volume support for libarchive. I'm hoping to get that into good shape this year. Your notes on size estimation will be quite helpful once I get further into that. Thanks again, and I may be in touch over the next few months with more questions.... ;-) Tim From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 05:42:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 799161065672 for ; Sun, 21 Feb 2010 05:42:05 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 3040A8FC08 for ; Sun, 21 Feb 2010 05:42:04 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o1L5gC5e011154; Sun, 21 Feb 2010 05:42:12 GMT (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id wm7hcc558u3iyztw7rsbg8prsa; Sun, 21 Feb 2010 05:42:11 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4B80C7E7.1090303@freebsd.org> Date: Sat, 20 Feb 2010 21:43:03 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090601 SeaMonkey/1.1.16 MIME-Version: 1.0 To: bf1783@gmail.com References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Alex RAY Subject: Re: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 05:42:05 -0000 b. f. wrote: >>> The code organization depends on what you want to do with it and how you >>> want to update the code in the future, if your lzma library is third party. >> LZMA made by Igor Pavlov, and since 4.62 it licensed under Public Domain. >> So we can use it, if we need. >> >>> If you never intend to update the lzma code then I guess it's fine to >>> embed it in a big .c file. For a port, it doesn't matter much since it >>> is your own thing. There are stricter rules on maintainability and style >>> if you want it in the base system. >> So, I think right place for lzma library under sys/contrib directory, if I "promise" >maintain it? > > There are already long-standing plans to import lzma compression > libraries into the base system, by the libarchive maintainers. Libarchive has hooks to support the liblzma library from Lasse Collin's "xz utils" package. When linked against this library, libarchive (and hence bsdtar) has full support for "bare" lzma streams and for Collin's "xz" format (which is a big improvement over bare lzma streams). Like Igor Pavlov's original code, the "xz utils" code is in the public domain. Officially, xz utils is in version 4.999.8beta; Collin has promised to bump the version to 5.0 once he feels the code is sufficiently stable. As far as I can tell, it's actually in pretty good shape now; the changes going in seem to mostly be minor portability fixes. I see no reason to not import the current version into the FreeBSD base system. I plan to do so but may not get to it very soon; I certainly would not complain if someone else beat me to it. ;-) There are other lzma libraries around, of course. In particular, I've seen references to the lzma libraries distributed with lzip. Unfortunately, lzip is GPLv3 so is not a candidate for inclusion in FreeBSD's base system. Tim From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 04:54:14 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72E6A106566B for ; Sun, 21 Feb 2010 04:54:14 +0000 (UTC) (envelope-from bf1783@googlemail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0F8E68FC16 for ; Sun, 21 Feb 2010 04:54:13 +0000 (UTC) Received: by wyb40 with SMTP id 40so95133wyb.13 for ; Sat, 20 Feb 2010 20:54:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:reply-to:date:message-id :subject:from:to:cc:content-type; bh=thMogT4Xri1EiTn3WTAiDnVSKJmmsNYnsocy1EzHBXY=; b=UEc93jAfh9cPkiOQj3c0HEE/Zu8lGV9RH5O4Dg1kN6epe0SjD6BlPGYw/RtZ9yJl3b UtSMUskdQDiuZumwWo5na7M0fBoZHi2w69MRVqsGJUVfwNekiE5zoFN+RcoHYOXShlvY /jcDfDD/cqQHRjUnMh5pCY9BVABpLCd+jZRgw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:reply-to:date:message-id:subject:from:to:cc :content-type; b=HILb/fcp652Wim9EwAZ07JaVIKl8JjLxyZyM18vGBqNL2upLIrCLZI9CasK3+dPTS6 wB02hzafCQeqnqtXHzrA8A5/oc7sRIr0D3qRFyPT4gbxzVcjWdWZZc2+ImfoMt/pmHhr cVtSL63C7vtPxJ/1QF6c2K0Hlsy3Qjtk1/c8c= MIME-Version: 1.0 Received: by 10.216.89.130 with SMTP id c2mr2730534wef.44.1266726222449; Sat, 20 Feb 2010 20:23:42 -0800 (PST) Date: Sat, 20 Feb 2010 23:23:42 -0500 Message-ID: From: "b. f." To: freebsd-hackers@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Sun, 21 Feb 2010 06:21:20 +0000 Cc: Alex RAY , kientzle@FreeBSD.org Subject: Re: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bf1783@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 04:54:14 -0000 >> The code organization depends on what you want to do with it and how you >> want to update the code in the future, if your lzma library is third party. > >LZMA made by Igor Pavlov, and since 4.62 it licensed under Public Domain. >So we can use it, if we need. > >> >> If you never intend to update the lzma code then I guess it's fine to >> embed it in a big .c file. For a port, it doesn't matter much since it >> is your own thing. There are stricter rules on maintainability and style >> if you want it in the base system. > >So, I think right place for lzma library under sys/contrib directory, if I "promise" >maintain it? There are already long-standing plans to import lzma compression libraries into the base system, by the libarchive maintainers. So you will probably have to coordinate with them. They have hooks in the current libarchive sources for native lzma compression, but they reportedly have been waiting for related file formats and compression methods to stabilize before enabling them by default. Pavlov has been making significant changes from release to release, so this may not be the easiest code to maintain. From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 19:38:16 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAB191065698 for ; Sun, 21 Feb 2010 19:38:16 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from gw03.mail.saunalahti.fi (gw03.mail.saunalahti.fi [195.197.172.111]) by mx1.freebsd.org (Postfix) with ESMTP id ADB5B8FC0A for ; Sun, 21 Feb 2010 19:38:16 +0000 (UTC) Received: from a91-153-117-195.elisa-laajakaista.fi (a91-153-117-195.elisa-laajakaista.fi [91.153.117.195]) by gw03.mail.saunalahti.fi (Postfix) with SMTP id 37EFB216729 for ; Sun, 21 Feb 2010 21:38:14 +0200 (EET) Date: Sun, 21 Feb 2010 21:38:13 +0200 From: Jaakko Heinonen To: freebsd-hackers@FreeBSD.org Message-ID: <20100221193813.GA21004@a91-153-117-195.elisa-laajakaista.fi> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Subject: apply(1) and libsbuf X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 19:38:17 -0000 Hi, I was looking to fix a bug (bin/95079) in apply(1) and found it easy to fix it using a sbuf buffer. Does anyone see a problem with adding libsbuf dependency for an utility such apply(1)? Nothing in usr.bin seems to use libsbuf currently. The patch: http://people.freebsd.org/~jh/patches/apply-sbuf.diff -- Jaakko From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 20:22:54 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52DB21065670 for ; Sun, 21 Feb 2010 20:22:54 +0000 (UTC) (envelope-from uqs@FreeBSD.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2001:470:9a47::1]) by mx1.freebsd.org (Postfix) with ESMTP id E44D98FC19 for ; Sun, 21 Feb 2010 20:22:53 +0000 (UTC) Received: from acme.spoerlein.net (localhost.spoerlein.net [IPv6:::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id o1LKMqdP005934 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 21 Feb 2010 21:22:52 +0100 (CET) (envelope-from uqs@FreeBSD.org) Received: (from uqs@localhost) by acme.spoerlein.net (8.14.4/8.14.4/Submit) id o1LKMphd005933; Sun, 21 Feb 2010 21:22:51 +0100 (CET) (envelope-from uqs@FreeBSD.org) Date: Sun, 21 Feb 2010 21:22:51 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Patrick Mahan , freebsd-hackers@FreeBSD.org Message-ID: <20100221202250.GF57731@acme.spoerlein.net> Mail-Followup-To: Patrick Mahan , freebsd-hackers@freebsd.org References: <4B800F60.60700@mahan.org> <20100220210314.GB22800@kiwi.sharlinx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100220210314.GB22800@kiwi.sharlinx.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Subject: Re: Building FreeBSD on a linux FC11 box. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 20:22:54 -0000 On Sat, 20.02.2010 at 13:03:14 -0800, R. Tyler Ballance wrote: > On Sat, 20 Feb 2010, Patrick Mahan wrote: > > Hopefully, this is not too ignorant a question. But has anyone every > > built the FreeBSD sources, both kernel and apps, on a linux platform? > > > > I did a google on 'cross-compile freebsd (linux)' and found (mostly) > > discussions regarding building the linux apps for FreeBSD, but not > > for building a FreeBSD kernel and world. There was a brief discussion > > on the mailing list back in 2005, but the poster never reported if > > he had success or failure. Also, there were discussions about building > > freebsd amd64 on a freebsd i386. > > > > I have done lots of development in linux using cross-compilers (mips > > embedded systems, powerpc, etc). But not with FreeBSD. > > > > I know I'll need to use 'bsdmake' no gnu 'make'. The other worry is > > kernel-toolchain target. I guess I'll just dive in and swim around > > and see what happens. > > You might want to ask the Debian GNU/kFreeBSD guys: > http://www.debian.org/ports/kfreebsd-gnu/ > > I bet they've got a good idea :) They are using the kernel only, though. Before you reinvent the wheel, take a look at NetBSD, they are known for their cross-compilation work. Uli From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 22:47:25 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8AA4C1065676 for ; Sun, 21 Feb 2010 22:47:25 +0000 (UTC) (envelope-from ray@ddteam.net) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.154]) by mx1.freebsd.org (Postfix) with ESMTP id 2763E8FC12 for ; Sun, 21 Feb 2010 22:47:24 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id e12so459447fga.13 for ; Sun, 21 Feb 2010 14:47:12 -0800 (PST) Received: by 10.87.66.14 with SMTP id t14mr14392544fgk.60.1266792431948; Sun, 21 Feb 2010 14:47:11 -0800 (PST) Received: from localhost (61-186-132-95.pool.ukrtel.net [95.132.186.61]) by mx.google.com with ESMTPS id 13sm1135046fxm.14.2010.02.21.14.47.09 (version=SSLv3 cipher=RC4-MD5); Sun, 21 Feb 2010 14:47:10 -0800 (PST) Date: Mon, 22 Feb 2010 00:46:56 +0200 From: Alex RAY To: Tim Kientzle Message-Id: <20100222004656.3dd2acfe.ray@ddteam.net> In-Reply-To: <4B80C7E7.1090303@freebsd.org> References: <4B80C7E7.1090303@freebsd.org> Organization: DDTeam.net X-Mailer: Sylpheed 2.7.1 (GTK+ 2.16.6; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: bf1783@gmail.com, freebsd-hackers@freebsd.org Subject: Re: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 22:47:25 -0000 On Sat, 20 Feb 2010 21:43:03 -0800 Tim Kientzle wrote: > b. f. wrote: > >>> The code organization depends on what you want to do with it and how you > >>> want to update the code in the future, if your lzma library is third party. > >> LZMA made by Igor Pavlov, and since 4.62 it licensed under Public Domain. > >> So we can use it, if we need. > >> > >>> If you never intend to update the lzma code then I guess it's fine to > >>> embed it in a big .c file. For a port, it doesn't matter much since it > >>> is your own thing. There are stricter rules on maintainability and style > >>> if you want it in the base system. > >> So, I think right place for lzma library under sys/contrib directory, if I "promise" >maintain it? > > > > There are already long-standing plans to import lzma compression > > libraries into the base system, by the libarchive maintainers. > > Libarchive has hooks to support the liblzma library > from Lasse Collin's "xz utils" package. When linked > against this library, libarchive (and hence bsdtar) > has full support for "bare" lzma streams and for > Collin's "xz" format (which is a big improvement over > bare lzma streams). Like Igor Pavlov's original > code, the "xz utils" code is in the public domain. > > Officially, xz utils is in version 4.999.8beta; Collin > has promised to bump the version to 5.0 once he feels > the code is sufficiently stable. As far as I can > tell, it's actually in pretty good shape now; the > changes going in seem to mostly be minor portability > fixes. I see no reason to not import the current > version into the FreeBSD base system. I plan > to do so but may not get to it very soon; I > certainly would not complain if someone else > beat me to it. ;-) > > There are other lzma libraries around, of course. > In particular, I've seen references to the lzma libraries > distributed with lzip. Unfortunately, lzip is GPLv3 > so is not a candidate for inclusion in FreeBSD's base > system. > > Tim > Thanks for that info. GEOM_ULZMA contain two parts mkulzma and GEOM module. libarchive can help for mkulzma utility, but how they can help for GEOM module? I need to know, where correct place for lzma code in kernel tree? -- Alex RAY From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 22:59:57 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20D8A1065676 for ; Sun, 21 Feb 2010 22:59:57 +0000 (UTC) (envelope-from eitanadlerlist@gmail.com) Received: from mail-fx0-f215.google.com (mail-fx0-f215.google.com [209.85.220.215]) by mx1.freebsd.org (Postfix) with ESMTP id 7EEC78FC0C for ; Sun, 21 Feb 2010 22:59:56 +0000 (UTC) Received: by fxm7 with SMTP id 7so1754961fxm.14 for ; Sun, 21 Feb 2010 14:59:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=PIAleDVSDxM/J6EqewLbdOkrl+59D91sqstG7fYMvJ0=; b=QB9WtvdoqzZJCPXARC6qgoPqgtzBJaspfKDGzygbm8PwRqUo0kqgovjxWMxgj9KecC 04EPdgy0GgHBWLcNDx0k5stG/OjFhcjqu9AETMeAKT4uF3bJz+UOUgGQheEl9/GVkZW7 siivDR35PkrNnyqCANsl2FyeREowLn6nXdVuA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; b=iBSevGx9K5DCubhxrt2YQTL4tsTgtJBmMTQEt2XiVZeYdSqfnE0/DEFQ0Z5wo4mRKT 0MlXHdc+Sfx3wMqGfxfICQeSQj12duG19K5cRMthFQxWC92erwdO5/YHwEoaSSoRSWHb 77GFmfzqjfuuEZvP2S+93ouAQ40i16vTa8GTk= MIME-Version: 1.0 Received: by 10.239.159.146 with SMTP id y18mr1505961hbc.196.1266793191380; Sun, 21 Feb 2010 14:59:51 -0800 (PST) In-Reply-To: <7d6fde3d1002181413p214389d3nd73798e597b9e26f@mail.gmail.com> References: <7d6fde3d1002181413p214389d3nd73798e597b9e26f@mail.gmail.com> From: Eitan Adler Date: Mon, 22 Feb 2010 00:59:29 +0200 Message-ID: To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, linimon@freebsd.org Subject: Re: junior tasks wiki page - please add stuff X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 22:59:57 -0000 On Fri, Feb 19, 2010 at 12:13 AM, Garrett Cooper wrote= : > On Thu, Feb 18, 2010 at 11:22 AM, Eitan Adler = wrote: >> I made a wiki page[1] to list small odd-jobs that beginners to work >> on. It is intended to answer the question "I'm just starting out with >> coding, what can I help with?" The tasks put on the wiki page should >> be things that require little programming skill. >> >> It would become a lot more useful if people added things to do. >> >> [1] http://wiki.freebsd.org/JuniorJobs > > =A0 =A01. Who should the wiki content `regulators' be (i.e. project-only > folks, outside folks, etc)? Anyone can sign up for an account and > because the information reflects on the project, it would be best > (IMO) if `proposals' // `tasks' were vetted by someone in the project > with sufficient say to prioritize the item. I could see a number of ways to deal with this. Ideally this list would be integrated with the PR database. However this requires a lot of time on the part of people with access to the PR database. Another possibility is just "mailing list proposals". People ask on -questions if something could be implemented which gets a lot of positive responses. Someone notices and adds it to the list. Alternatively it could be used as a semi request - not everything the list will make it in - it would just be a starting point. > =A0 =A02. Would it be a good idea to also provide a list of tasks that > are non-coding related (or at least have a smaller scope) so that > others could assist with in the project to help free up resources? We have http://wiki.freebsd.org/WantedPorts for ports For instance, some folks (like linimon@) really need a junior person (I'v= e > debated whether or not I should do it, but I don't want to assume too > much responsibility) to do semi-menial tasks like bug triage, > feedback, basic administration items, etc to free up Mark's time. That > way it would allow someone with less background and exposure to the > project to become more integrated with the project, outside of doing > an SoC project for instance. Agreed - I think that doing some small project on this list might be a good way to start. > =A0 =A0Just curious... > Thanks, > -Garrett > From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 23:00:08 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1395610656CE for ; Sun, 21 Feb 2010 23:00:07 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id C18678FC16 for ; Sun, 21 Feb 2010 23:00:07 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o1LN09Eu018511; Sun, 21 Feb 2010 23:00:09 GMT (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id yzqfzv9vf3bhqciv54ivjpn376; Sun, 21 Feb 2010 23:00:09 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4B81BB2D.2090009@freebsd.org> Date: Sun, 21 Feb 2010 15:01:01 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090601 SeaMonkey/1.1.16 MIME-Version: 1.0 To: Alex RAY References: <4B80C7E7.1090303@freebsd.org> <20100222004656.3dd2acfe.ray@ddteam.net> In-Reply-To: <20100222004656.3dd2acfe.ray@ddteam.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: bf1783@gmail.com, freebsd-hackers@freebsd.org Subject: Re: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 23:00:08 -0000 >> I see no reason to not import the current >> version [of liblzma and xz] into the FreeBSD base system. I plan >> to do so but may not get to it very soon; I >> certainly would not complain if someone else >> beat me to it. ;-) > > Thanks for that info. > GEOM_ULZMA contain two parts mkulzma and GEOM module. > libarchive can help for mkulzma utility, but how they can help for GEOM module? > I need to know, where correct place for lzma code in kernel tree? I think you mean "liblzma" can help. I plan to import liblzma into FreeBSD base system as /usr/lib/liblzma.a and /usr/lib/liblzma.so (that may change to /lib/liblzma.so), with the headers in /usr/include/. I expect xz to end up as /usr/bin/xz. I'm not familiar enough with kernel development processes to understand the implications there. Cheers, Tim From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 21 23:20:18 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00E59106566C for ; Sun, 21 Feb 2010 23:20:18 +0000 (UTC) (envelope-from a_best01@uni-muenster.de) Received: from zivm-exrelay1.uni-muenster.de (ZIVM-EXRELAY1.UNI-MUENSTER.DE [128.176.192.14]) by mx1.freebsd.org (Postfix) with ESMTP id 729598FC12 for ; Sun, 21 Feb 2010 23:20:16 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.49,513,1262559600"; d="scan'208";a="297082749" Received: from zivmaildisp1.uni-muenster.de (HELO ZIVMAILUSER01.UNI-MUENSTER.DE) ([128.176.188.85]) by zivm-relay1.uni-muenster.de with ESMTP; 22 Feb 2010 00:20:15 +0100 Received: by ZIVMAILUSER01.UNI-MUENSTER.DE (Postfix, from userid 149459) id 5B0451B0768; Mon, 22 Feb 2010 00:20:15 +0100 (CET) Date: Mon, 22 Feb 2010 00:20:14 +0100 (CET) From: Alexander Best Sender: Organization: Westfaelische Wilhelms-Universitaet Muenster To: Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: `make cleanworld` and /usr/obj/lib32/usr/src X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 23:20:18 -0000 hi there, anybody fancy the idea of including /usr/obj/lib32/usr/src in the cleanworld target on amd64? cheers. alex From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 01:16:38 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96DFB1065692 for ; Mon, 22 Feb 2010 01:16:38 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 6830F8FC0C for ; Mon, 22 Feb 2010 01:16:38 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o1M1GkbM019713; Mon, 22 Feb 2010 01:16:46 GMT (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id wk34rd24n52t4c6sqpatdq5v96; Mon, 22 Feb 2010 01:16:45 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4B81DB31.70201@freebsd.org> Date: Sun, 21 Feb 2010 17:17:37 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090601 SeaMonkey/1.1.16 MIME-Version: 1.0 To: Juergen Lock References: <20100217215940.GA19713@triton8.kn-bremen.de> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> <201002200100.48161.jkim@FreeBSD.org> <20100220103329.GA35467@triton8.kn-bremen.de> In-Reply-To: <20100220103329.GA35467@triton8.kn-bremen.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Duane.Hesser@gmail.com Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 01:16:38 -0000 Juergen, Could you try the current version of read_open_filename from: http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_filename.c You should be able to just copy it into your FreeBSD source tree and recompile. Duane Hesser's comments clarified for me that disk and tape I/O do require fundamentally different strategies, so I refactored this code to clearly break out those strategies. This in turn allowed me to do some straightforward I/O optimization of the disk case, which sped up your example by about 100x. I have a 2G DVD here on which "tar tvf /dev/acd0" took 90s originally, takes 14s with your patch, and now takes only 0.6s. For comparison, after mounting the disk, "find /mnt" takes 1.6s (which drops to .1s after the cache is warmed). It would be interesting to also compare two ways of copying all of the files: "tar xvf /dev/acd0" on an unmounted disk and "cp -R" on a mounted disk. The non-disk cases are all still handled generically, but there are now clearly-labeled holes where someone could add optimized I/O strategies for tapes, sockets, etc. I've even included a number of TODO comments about what strategies I think are worth exploring, including tricks like using MTIO ioctls on tape, mmap() for disk, and nonblocking I/O for sockets and pipes. I expect any one of these is around a dozen lines of code, so should be within the reach of even fairly junior developers. Tim From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 11:03:04 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55E5E106566C for ; Mon, 22 Feb 2010 11:03:04 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout3.freenet.de (mout3.freenet.de [IPv6:2001:748:100:40::2:5]) by mx1.freebsd.org (Postfix) with ESMTP id E68B88FC13 for ; Mon, 22 Feb 2010 11:03:03 +0000 (UTC) Received: from [195.4.92.15] (helo=5.mx.freenet.de) by mout3.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #1) id 1NjW46-00006r-JE; Mon, 22 Feb 2010 12:03:02 +0100 Received: from p57ae04a0.dip0.t-ipconnect.de ([87.174.4.160]:56346 helo=ernst.jennejohn.org) by 5.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #1) id 1NjW46-0007nn-AZ; Mon, 22 Feb 2010 12:03:02 +0100 Date: Mon, 22 Feb 2010 12:03:00 +0100 From: Gary Jennejohn To: Alexander Best Message-ID: <20100222120300.22d4efb1@ernst.jennejohn.org> In-Reply-To: References: X-Mailer: Claws Mail 3.7.4 (GTK+ 2.16.2; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org Subject: Re: `make cleanworld` and /usr/obj/lib32/usr/src X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 11:03:04 -0000 On Mon, 22 Feb 2010 00:20:14 +0100 (CET) Alexander Best wrote: > hi there, > > anybody fancy the idea of including /usr/obj/lib32/usr/src in the cleanworld > target on amd64? > Seems like a reasonable thing to do. --- Gary Jennejohn From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 12:04:20 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0427A1065670 for ; Mon, 22 Feb 2010 12:04:20 +0000 (UTC) (envelope-from joel@FreeBSD.org) Received: from mail.vnode.se (mail.vnode.se [62.119.52.80]) by mx1.freebsd.org (Postfix) with ESMTP id B38508FC12 for ; Mon, 22 Feb 2010 12:04:19 +0000 (UTC) Received: from mail.vnode.se (localhost [127.0.0.1]) by mail.vnode.se (Postfix) with ESMTP id 0CE03E3F086; Mon, 22 Feb 2010 13:04:18 +0100 (CET) X-Virus-Scanned: amavisd-new at vnode.se Received: from mail.vnode.se ([127.0.0.1]) by mail.vnode.se (mail.vnode.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zacLlk6WCI1e; Mon, 22 Feb 2010 13:04:15 +0100 (CET) Received: by mail.vnode.se (Postfix, from userid 1004) id 7F5B0E3F085; Mon, 22 Feb 2010 13:04:15 +0100 (CET) Date: Mon, 22 Feb 2010 13:04:15 +0100 From: Joel Dahl To: "Pedro F. Giffuni" Message-ID: <20100222120415.GA91890@mail.vnode.se> References: <346566.86917.qm@web113519.mail.gq1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <346566.86917.qm@web113519.mail.gq1.yahoo.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@FreeBSD.org Subject: Re: Anyone updating the Project Ideas page? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 12:04:20 -0000 On 7:49 19-02-10, Pedro F. Giffuni wrote: > Hello; > > I've sent some private messages before trying to get > some updates to the Project ideas page > http://www.freebsd.org/projects/ideas/ideas.html I'd like to see the whole thing moved to the wiki. -- Joel From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 15:44:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 926871065692 for ; Mon, 22 Feb 2010 15:44:03 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505e.appriver.com [98.129.35.9]) by mx1.freebsd.org (Postfix) with ESMTP id 5B3A18FC19 for ; Mon, 22 Feb 2010 15:44:02 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.14 X-Note-Reverse-DNS: ht01.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.14] (HELO ht01.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 25052201 for freebsd-hackers@freebsd.org; Mon, 22 Feb 2010 09:44:00 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by ht01.exg5.exghost.com ([98.129.23.14]) with mapi; Mon, 22 Feb 2010 09:44:02 -0600 From: Peter Steele To: "freebsd-hackers@freebsd.org" Date: Mon, 22 Feb 2010 09:44:01 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: AcqyIJlpWWkdyL4NQ1GPkZbhrycV7wBsHOSQ Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> In-Reply-To: <20100220113349.GA22800@kiwi.sharlinx.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 15:44:03 -0000 >Just out of curiosity, can you attach to the process via gdb and get a bac= ktrace? This smells like a locked pthread_join I hit in my own code a few w= eeks ago I'm not using the debug version of ntpd so the backtrace isn't too useful, = but here's what I get: (gdb) bt #0 0x0000000800d52bfc in select () from /lib/libc.so.7 #1 0x0000000000425273 in ?? () #2 0x000000000040540e in ?? () #3 0x0000000800580000 in ?? () #4 0x0000000000000000 in ?? () The trace continues for 700+ entries. The first entry is useful enough thou= gh. One of the parameters to select() is a timeout parameter. Every time I = do the backtrace it's stuck on this select call so it seems they have an in= finite timeout set. One of these was running all weekend in fact and it's s= till stuck. Curiously, this problem only happens when we make the call from= code via a system() call. If I run the same command interactively, it neve= r hangs: # /usr/sbin/ntpd -g -q ntpd: time set +28845.997063s The same code that runs this command does not hang when we run it on a BSD = 7 box.=20 I think I'm going to have to build the debug version of ntpd and try to deb= ug it. Definitely something weird going on. From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 15:52:01 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F0AF106566B for ; Mon, 22 Feb 2010 15:52:01 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.25]) by mx1.freebsd.org (Postfix) with ESMTP id A55E48FC0C for ; Mon, 22 Feb 2010 15:52:00 +0000 (UTC) Received: by ey-out-2122.google.com with SMTP id 9so594128eyd.9 for ; Mon, 22 Feb 2010 07:51:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=mgh3lYjIBtLh7gQlqCnqxRwaJewvm5pZ6A/NG5/TDuk=; b=sgQlyBI6OULEcnq810dr7m7wdXhNOoTSrHMgR/VNLnDxnr7zXZucFe3XneLVly0qOs VlJZ6ReX3tC2OACng6GXE+HKipIvn437zC/PGuSa6GYLDnToXktIGEy0F/ufRPQPS/Oa T8z+FOc8TqMD+y7PCArFzwUp5ax/nB0yKg5qE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=t8qgkkHnK5KyqCFcktdJWr4Gf8HnbIfk/fwcRbk5VqLojp80OpbqkEaF1Yp840NvV7 BPu2NjsFv2SlDISTsOl0v+yD1aStSZdLjGAn8/vSgZi0cmWbiG9tnmyxg2ZO35ZPL6Vd SY5MGRQkMbkZj4DYHwsLhygXJq3TrA+6VDy8g= MIME-Version: 1.0 Received: by 10.213.100.208 with SMTP id z16mr4382895ebn.80.1266853916562; Mon, 22 Feb 2010 07:51:56 -0800 (PST) In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> Date: Mon, 22 Feb 2010 10:51:56 -0500 Message-ID: From: Ryan Stone To: Peter Steele Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 15:52:01 -0000 You're going to need a debug version of libc, too. gdb won't be able to find a backtrace out of a libc function without it. From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 16:01:48 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A2C41065696 for ; Mon, 22 Feb 2010 16:01:48 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505c.appriver.com [98.129.35.7]) by mx1.freebsd.org (Postfix) with ESMTP id 60D898FC26 for ; Mon, 22 Feb 2010 16:01:48 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.14 X-Note-Reverse-DNS: ht01.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.14] (HELO ht01.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 28917680 for freebsd-hackers@freebsd.org; Mon, 22 Feb 2010 10:01:46 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by ht01.exg5.exghost.com ([98.129.23.14]) with mapi; Mon, 22 Feb 2010 10:01:47 -0600 From: Peter Steele To: "freebsd-hackers@freebsd.org" Date: Mon, 22 Feb 2010 10:01:46 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acqz1wAQdYoZ2dInQG2e7o15W4OUdwAAMdUw Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 16:01:48 -0000 >You're going to need a debug version of libc, too. gdb won't be able to f= ind a backtrace out of a libc function without it. Yeah, you're right. This is definitely an annoying bug... From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 14:48:42 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6E24106566B for ; Mon, 22 Feb 2010 14:48:42 +0000 (UTC) (envelope-from giffunip@tutopia.com) Received: from web113512.mail.gq1.yahoo.com (web113512.mail.gq1.yahoo.com [98.136.167.52]) by mx1.freebsd.org (Postfix) with SMTP id 842468FC12 for ; Mon, 22 Feb 2010 14:48:41 +0000 (UTC) Received: (qmail 94030 invoked by uid 60001); 22 Feb 2010 14:48:37 -0000 Message-ID: <16886.92863.qm@web113512.mail.gq1.yahoo.com> X-YMail-OSG: XUzxBfgVM1kND3eSje.Pk7d8DvcIFYb865Rzx7ePGdxebYa7SFvcpnPtL.DK7Rf033AHpc9s8P3PEga5.l7vMuvDdW2yJWjYXJ8zUs72bZ7IrhuB1yvOluSYFyzI4h5.MVaZU6pFiXHSmmULTcH1tQsYoDp5pIR8uGobD4Dr7ud_pu9bao6ObQeHas6Ik0oYNHUWYTNJXiLgPuQMpJNzoOLYabZYqIEK1h3nQny0qQp3fSgoVTSrLnHKQBzQ5AHfDT0cVUHAzY6IF4tOE99jij4frST2N1ey4uCrc8wIafLMAfpDBrL.jm23_gazx2hypB2E7G4KWSB6Ha7XWerrS9tfwOCr2C0b0W817Dj5S6nA7tyGF.S76kH63A-- Received: from [190.157.123.47] by web113512.mail.gq1.yahoo.com via HTTP; Mon, 22 Feb 2010 06:48:36 PST X-RocketYMMF: giffunip X-Mailer: YahooMailRC/300.3 YahooMailWebService/0.8.100.260964 References: <346566.86917.qm@web113519.mail.gq1.yahoo.com> <20100222120415.GA91890@mail.vnode.se> Date: Mon, 22 Feb 2010 06:48:36 -0800 (PST) From: "Pedro F. Giffuni" To: Joel Dahl In-Reply-To: <20100222120415.GA91890@mail.vnode.se> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailman-Approved-At: Mon, 22 Feb 2010 16:42:54 +0000 Cc: freebsd-hackers@FreeBSD.org Subject: Re: Anyone updating the Project Ideas page? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 14:48:42 -0000 ----- Original Message ---- ... > > On 7:49 19-02-10, Pedro F. Giffuni wrote: > > Hello; > > > > I've sent some private messages before trying to get > > some updates to the Project ideas page > > http://www.freebsd.org/projects/ideas/ideas.html > > > I'd like to see the whole thing moved to the wiki. > While I like the idea, as the website would be better maintained, I have to say in actual form the page looks better than in the wiki. Just my 0.02$, Pedro. From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 17:52:11 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78E47106568D; Mon, 22 Feb 2010 17:52:11 +0000 (UTC) (envelope-from sarawgi.aditya@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 410EE8FC23; Mon, 22 Feb 2010 17:52:11 +0000 (UTC) Received: by pwj7 with SMTP id 7so3371917pwj.13 for ; Mon, 22 Feb 2010 09:52:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=/Vk0ca9fD36+zQS9Ge64Qvx+vybuTd+HYh9BzXj2ehQ=; b=jcrr8ynFpcuQwKLAVpQVq+GrhsohnOjfH+eMzYRUuQqsODD9ku7NHhvW9720pVTorr k9gR/h5LbtX3EEQ6X6dLxMwssauOokbL86nVKhV3oykiBNzIZ/tFRfqZ21zVe4SpGRCh 2X0hsxAoh12wZamT5GGxrL26b2+nHWcFfFgHo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=m5W6lxQJwQCsFmskYYO0vrjj10Zo2V5eesRpsNb7g7SrNb0Zdd67xpf2jarLeAnSt/ 7mLjFv9E+IpumNTIbwgq5mS4sJXHjOx4sbY5w4DEq7Uo7HNnS+LQdua5r3tyVnbiRFUg l5G0uLw4ci1roJGSWsRkCiUb1Yx6c2ciOp+T0= Received: by 10.141.213.22 with SMTP id p22mr424118rvq.94.1266859812639; Mon, 22 Feb 2010 09:30:12 -0800 (PST) Received: from ([183.87.28.148]) by mx.google.com with ESMTPS id 23sm125189pzk.10.2010.02.22.09.30.10 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 22 Feb 2010 09:30:11 -0800 (PST) Date: Mon, 22 Feb 2010 23:00:31 +0000 From: Aditya Sarawgi To: "Pedro F. Giffuni" Message-ID: <4b82bf23.9713f30a.7dd8.0766@mx.google.com> References: <346566.86917.qm@web113519.mail.gq1.yahoo.com> <20100222120415.GA91890@mail.vnode.se> <16886.92863.qm@web113512.mail.gq1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <16886.92863.qm@web113512.mail.gq1.yahoo.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: hackers@FreeBSD.org, joel@FreeBSD.org Subject: Re: Anyone updating the Project Ideas page? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 17:52:11 -0000 On Mon, Feb 22, 2010 at 06:48:36AM -0800, Pedro F. Giffuni wrote: > > > ----- Original Message ---- > ... > > > > On 7:49 19-02-10, Pedro F. Giffuni wrote: > > > Hello; > > > > > > I've sent some private messages before trying to get > > > some updates to the Project ideas page > > > http://www.freebsd.org/projects/ideas/ideas.html > > > > > > I'd like to see the whole thing moved to the wiki. > > > > While I like the idea, as the website would be better > maintained, I have to say in actual form the page > looks better than in the wiki. > > > Just my 0.02$, > > Pedro. > +1 With gsoc just around the corner I think the project ideas page should be updated and moving the whole thing to wiki or maintaining a similar page on wiki(along with the original page) sounds good to me. > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Aditya Sarawgi From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 20:44:52 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9B63106566B for ; Mon, 22 Feb 2010 20:44:52 +0000 (UTC) (envelope-from eitanadlerlist@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 53A678FC08 for ; Mon, 22 Feb 2010 20:44:51 +0000 (UTC) Received: by fxm23 with SMTP id 23so2899680fxm.3 for ; Mon, 22 Feb 2010 12:44:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=6g+AVa787hO7cNHieyKHf5n4rqNAICti84IbJ9wJC98=; b=gtb6gWeLZjAy+TSJnVrfV9/WIKOAxGtpxiYH3PaEjpbos6UYw7/jnHuy7v10n5Fo+Z MK+77fqNNcjG1dsSn4kqPABqrFSncDLV5jGmY66MN4orR8AJQ4+hN6rYftOfdqisSpeS yo9Dq4/EL6ERhstB2J6ofDJF9eZ2PNJogt7zQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; b=fPorrq3Nbmj9KGq7Hr3cXDZUbxxYVvTa5xUFqvAOulHX8OU5Mk66mqYRb0B7sq4gId ekBnr1Gwo9tUgzko4hnTkbdtTM4x2L/9VkwyCdSY4D7u++MPVgdaFX0jI5D0Jnqx7ynj 86zM7xjzdMxmtPcSP9KPkdRbX7DY9/gEnocY0= MIME-Version: 1.0 Received: by 10.239.161.76 with SMTP id g12mr1668825hbd.182.1266871484182; Mon, 22 Feb 2010 12:44:44 -0800 (PST) In-Reply-To: <4b82bf23.9713f30a.7dd8.0766@mx.google.com> References: <346566.86917.qm@web113519.mail.gq1.yahoo.com> <20100222120415.GA91890@mail.vnode.se> <16886.92863.qm@web113512.mail.gq1.yahoo.com> <4b82bf23.9713f30a.7dd8.0766@mx.google.com> From: Eitan Adler Date: Mon, 22 Feb 2010 22:44:24 +0200 Message-ID: To: Aditya Sarawgi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, "Pedro F. Giffuni" , joel@freebsd.org Subject: Re: Anyone updating the Project Ideas page? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 20:44:53 -0000 On Tue, Feb 23, 2010 at 1:00 AM, Aditya Sarawgi wrote: > On Mon, Feb 22, 2010 at 06:48:36AM -0800, Pedro F. Giffuni wrote: >> >> >> ----- Original Message ---- >> ... >> > >> > On =A07:49 19-02-10, Pedro F. Giffuni wrote: >> > > Hello; >> > > >> > > I've sent some private messages before trying to get >> > > some updates to the Project ideas page >> > > http://www.freebsd.org/projects/ideas/ideas.html >> > >> > >> > I'd like to see the whole thing moved to the wiki. >> > >> >> While I like the idea, as the website would be better >> maintained, I have to say in actual form the page >> looks better than in the wiki. >> >> >> Just my 0.02$, >> >> Pedro. >> > > +1 > With gsoc just around the corner I think the project ideas page should > be updated and moving the whole thing to wiki or maintaining a similar > page on wiki(along with the original page) =A0sounds good to me. > I'll take the bait and make the page on the wiki. However someone else will have to update the doc source to include a link.... From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 22 23:06:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 402301065694; Mon, 22 Feb 2010 23:06:35 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id C45828FC1F; Mon, 22 Feb 2010 23:06:34 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id E3E861E0011B; Tue, 23 Feb 2010 00:06:32 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1MN3BBW020098; Tue, 23 Feb 2010 00:03:11 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1MN3BuF020097; Tue, 23 Feb 2010 00:03:11 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Tue, 23 Feb 2010 00:03:11 +0100 To: Tim Kientzle Message-ID: <20100222230311.GA19535@triton8.kn-bremen.de> References: <20100217215940.GA19713@triton8.kn-bremen.de> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> <201002200100.48161.jkim@FreeBSD.org> <20100220103329.GA35467@triton8.kn-bremen.de> <4B81DB31.70201@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B81DB31.70201@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Mon, 22 Feb 2010 23:20:17 +0000 Cc: freebsd-hackers@freebsd.org, Juergen Lock , Duane.Hesser@gmail.com Subject: Re: "tar tfv /dev/cd0" speedup patch; Linux compatibility patches X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 23:06:35 -0000 On Sun, Feb 21, 2010 at 05:17:37PM -0800, Tim Kientzle wrote: > Juergen, > > Could you try the current version of read_open_filename from: > > http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_filename.c > > You should be able to just copy it into your FreeBSD source > tree and recompile. > Small but important bug: Index: archive_read_open_filename.c @@ -94,7 +94,7 @@ archive_read_open_filename(struct archiv struct read_file_data *mine; void *buffer; int fd; - int is_disk_like; + int is_disk_like = 0; #if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) off_t mediasize = 0; #elif defined(__NetBSD__) || defined(__OpenBSD__) > Duane Hesser's comments clarified for me that disk and tape > I/O do require fundamentally different strategies, so I > refactored this code to clearly break out those > strategies. This in turn allowed me to do some > straightforward I/O optimization of the disk case, > which sped up your example by about 100x. I have a 2G > DVD here on which "tar tvf /dev/acd0" took 90s originally, > takes 14s with your patch, and now takes only 0.6s. > For comparison, after mounting the disk, "find /mnt" > takes 1.6s (which drops to .1s after the cache is warmed). > It would be interesting to also compare two ways of > copying all of the files: "tar xvf /dev/acd0" > on an unmounted disk and "cp -R" on a mounted disk. > > The non-disk cases are all still handled generically, but > there are now clearly-labeled holes where someone could > add optimized I/O strategies for tapes, sockets, etc. > I've even included a number of TODO comments about > what strategies I think are worth exploring, including > tricks like using MTIO ioctls on tape, mmap() for disk, > and nonblocking I/O for sockets and pipes. I expect > any one of these is around a dozen lines of code, so > should be within the reach of even fairly junior > developers. Btw... the lseek + read combinations could also be replaced with pread(2), for the least amount of syscalls. :) (Especially since I noticed this version at least when ran on an optical disc with iso9660 ends up doing lots of lseek()s towards the end without ever reading anything from the disc anymore...) And to test the above fix, I ran the Linux-built bsdtar in the linuxolator, and after confirming it now does read all of the disc again I went and patched the remaining compatibility problems, i.e. I now have disks appear as block devices for Linux processes (there already was commented out code for that in linux_stats.c, I hope my version is now `correct enough' to be usable [1]), and I made a simple patch to make lseek SEEK_END (L_XTND in the source) dtrt on disk devices too by simply invoking the DIOCGMEDIASIZE ioctl there. [2] Patches are here: (made on stable/8, if they don't apply on head I'll have to make extra versions for that...) http://people.freebsd.org/~nox/linuxdisk-blk.patch [1] http://people.freebsd.org/~nox/lseek-seek_end.patch [2] And yes, with these patches the Linux bsdtar now also runs fast on FreeBSD. :) Cheers, Juergen From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 23 04:46:53 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 394D51065672 for ; Tue, 23 Feb 2010 04:46:53 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id E000F8FC13 for ; Tue, 23 Feb 2010 04:46:52 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o1N4l09J034635; Tue, 23 Feb 2010 04:47:00 GMT (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id gsxwrzan89ie9evedkbkapzdca; Tue, 23 Feb 2010 04:47:00 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4B835DFD.5060507@freebsd.org> Date: Mon, 22 Feb 2010 20:47:57 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090601 SeaMonkey/1.1.16 MIME-Version: 1.0 To: Juergen Lock References: <20100217215940.GA19713@triton8.kn-bremen.de> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> <201002200100.48161.jkim@FreeBSD.org> <20100220103329.GA35467@triton8.kn-bremen.de> <4B81DB31.70201@freebsd.org> <20100222230311.GA19535@triton8.kn-bremen.de> In-Reply-To: <20100222230311.GA19535@triton8.kn-bremen.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: "tar tfv /dev/cd0" speedup patch; Linux compatibility patches X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 04:46:53 -0000 Juergen Lock wrote: > On Sun, Feb 21, 2010 at 05:17:37PM -0800, Tim Kientzle wrote: >> >> Could you try the current version of read_open_filename from: >> >> http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_filename.c > > Small but important bug: > > Index: archive_read_open_filename.c > @@ -94,7 +94,7 @@ archive_read_open_filename(struct archiv > struct read_file_data *mine; > void *buffer; > int fd; > - int is_disk_like; > + int is_disk_like = 0; Ah. Good catch! > Btw... the lseek + read combinations could also be replaced with > pread(2), for the least amount of syscalls. :) Unfortunately, pread() isn't as portable. It's an interesting idea, though; maybe I'll play with that. > (Especially since I > noticed this version at least when ran on an optical disc with iso9660 > ends up doing lots of lseek()s towards the end without ever reading > anything from the disc anymore...) Fortunately, because there are no reads, those lseek() requests are pretty cheap. But you're right, it wouldn't be hard to eliminate the unnecessary system calls. > Patches are here: (made on stable/8, if they don't apply on head > I'll have to make extra versions for that...) > http://people.freebsd.org/~nox/linuxdisk-blk.patch [1] > http://people.freebsd.org/~nox/lseek-seek_end.patch [2] I'll let someone more familiar with the linuxolator code comment on those. > And yes, with these patches the Linux bsdtar now also runs fast > on FreeBSD. :) Excellent! Thanks so much for your help. If you have any further ideas (or patches ;-) for improving that libarchive code, please let me know. Cheers, Tim From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 23 07:34:15 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10832106566C for ; Tue, 23 Feb 2010 07:34:15 +0000 (UTC) (envelope-from shrivatsan_v@yahoo.com) Received: from web112005.mail.gq1.yahoo.com (web112005.mail.gq1.yahoo.com [67.195.23.83]) by mx1.freebsd.org (Postfix) with SMTP id C07AD8FC08 for ; Tue, 23 Feb 2010 07:34:14 +0000 (UTC) Received: (qmail 4917 invoked by uid 60001); 23 Feb 2010 07:34:12 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1266910451; bh=YsiuL1/DOJWtnRa2eFzzjIE19mnLk3nczNiTZoO1qVg=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type; b=gmTk+GF9/nz5rVF306WkKtIap0igroMVesJCSTs9LsoXqskRagdR9AJApcl/ruuepxlwV682VMIQT0iQlzW9ZIv3AZeG13g8s7UKSALqRO5vu0wp3KOGXqaH1/lcSj5gQGpC6XT67ev7wR6bzYHDBWa+UJ8ZXd2Sk5FvezXjRZE= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type; b=XFdPZ0emBecnPfzEj4kOiy87W8ffyWwMtMlVa8Soe+97euJS6vDnGESUEXw9krw9J9RtKEdb0Z4PvNYOQfmwgIR8BA0YPU7eDVJwDIoIWGZDalRhiT4AnbcOQ7d8JrCvNM7WAk2joCtORScMNa4SvsCjNj694KlFvsebhq0/Zug=; Message-ID: <986041.3700.qm@web112005.mail.gq1.yahoo.com> X-YMail-OSG: AGyr0okVM1kEBtW.w_g7H_rHzJCgph4.NPCpIrXp9cB6DPeLLfD0SdLh1QSK33aUUtYmBRnXowuqhnLE93RU.1ElODDBmACqO9uy99sXrgf_UBMVrhSlsn69t39SvbxvnbD44vEocViP_1ezGyTsuw5JucYZdMCNEKbWJsWKHFWVQAx7xCB7KPRPXIzKadEp00p7Ot0SzlZZ.4JEC4.pHWcLq6MnrBCiG78Z3r.1R9zdCRQxNctvkgTaWaNLzDFKHWkyZctDV2O196Q.t7lIvw-- Received: from [66.129.224.36] by web112005.mail.gq1.yahoo.com via HTTP; Mon, 22 Feb 2010 23:34:11 PST X-Mailer: YahooMailClassic/9.2.12 YahooMailWebService/0.8.100.260964 Date: Mon, 22 Feb 2010 23:34:11 -0800 (PST) From: Shrivatsan To: freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: shrivatsan@gmail.com Subject: sched_lock mutex and sleepq_wait X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 07:34:15 -0000 Hi, I am trying to understand how msleep() routine suspends the currently executing thread. I see that msleep() calls sleepq_wait(). What I don't understand here is the way in which sched_lock mutex is handled. I took the following snippet from FreeBSD 6: sleepq_wait(void *wchan) { MPASS(!(curthread->td_flags & TDF_SINTR)); mtx_lock_spin(&sched_lock); sleepq_switch(wchan); mtx_unlock_spin(&sched_lock); } sched_lock mutex is held, and sleepq_switch() eventually calls cpu_switch() which switches to a new thread. I don't exactly understand when the sched_lock mutex is released. Can someone please help me? Thanks, -shrivatsan From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 23 12:44:39 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5DE3106566C for ; Tue, 23 Feb 2010 12:44:39 +0000 (UTC) (envelope-from ray@dlink.ua) Received: from dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id 632CC8FC0A for ; Tue, 23 Feb 2010 12:44:39 +0000 (UTC) Received: from gw ([192.168.10.10] helo=terran) by dlink.ua with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1NjthG-0005xL-7I for freebsd-hackers@FreeBSD.org; Tue, 23 Feb 2010 14:17:02 +0200 Date: Tue, 23 Feb 2010 14:03:28 +0200 From: Alexandr Rybalko To: freebsd-hackers@FreeBSD.org Message-Id: <20100223140328.50aa08f1.ray@dlink.ua> Organization: D-Link X-Mailer: Sylpheed 2.7.1 (GTK+ 2.16.6; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: hastd without openssl X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 12:44:39 -0000 Hi, I make small patch for no build hastd without openssl. Index: sbin/Makefile =================================================================== --- sbin/Makefile (revision 204220) +++ sbin/Makefile (working copy) @@ -37,7 +37,7 @@ SUBDIR= adjkerntz \ growfs \ gvinum \ hastctl \ - hastd \ + ${_hastd} \ ifconfig \ init \ ${_ipf} \ @@ -121,6 +121,10 @@ _rtsol= rtsol _bsdlabel= bsdlabel .endif +.if ${MK_OPENSSL} != "no" +_hastd= hastd +.endif + .if ${MK_QUOTAS} != "no" _quotacheck= quotacheck .endif -- Alexandr Rybalko aka Alex RAY From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 23 17:04:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA3FF106566C for ; Tue, 23 Feb 2010 17:04:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id BC75E8FC14 for ; Tue, 23 Feb 2010 17:04:35 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 6D20E46B35; Tue, 23 Feb 2010 12:04:35 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 296A28A025; Tue, 23 Feb 2010 12:04:34 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Tue, 23 Feb 2010 11:09:42 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20100120; KDE/4.3.1; amd64; ; ) References: <986041.3700.qm@web112005.mail.gq1.yahoo.com> In-Reply-To: <986041.3700.qm@web112005.mail.gq1.yahoo.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002231109.42603.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 23 Feb 2010 12:04:34 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.3 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: shrivatsan@gmail.com, Shrivatsan Subject: Re: sched_lock mutex and sleepq_wait X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 17:04:36 -0000 On Tuesday 23 February 2010 2:34:11 am Shrivatsan wrote: > Hi, > > I am trying to understand how msleep() routine suspends the currently > executing thread. I see that msleep() calls sleepq_wait(). > > What I don't understand here is the way in which sched_lock mutex is > handled. > > I took the following snippet from FreeBSD 6: > > sleepq_wait(void *wchan) > { > MPASS(!(curthread->td_flags & TDF_SINTR)); > mtx_lock_spin(&sched_lock); > sleepq_switch(wchan); > mtx_unlock_spin(&sched_lock); > } > > sched_lock mutex is held, and sleepq_switch() eventually calls > cpu_switch() which switches to a new thread. > > I don't exactly understand when the sched_lock mutex is released. > > Can someone please help me? mi_switch() changes the owner of sched_lock to hand it off to the new thread during the switch. The new thread then returns from mi_switch() and eventually returns to something like sleepq_wait() where it unlocks sched_lock. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 23 19:12:21 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 831B21065676 for ; Tue, 23 Feb 2010 19:12:21 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-9.mx.aerioconnect.net [216.240.47.69]) by mx1.freebsd.org (Postfix) with ESMTP id 671438FC1F for ; Tue, 23 Feb 2010 19:12:21 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o1NJCKXQ031244; Tue, 23 Feb 2010 11:12:20 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 4ABA92D6019; Tue, 23 Feb 2010 11:12:20 -0800 (PST) Message-ID: <4B8428C7.8010705@elischer.org> Date: Tue, 23 Feb 2010 11:13:11 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Shrivatsan References: <986041.3700.qm@web112005.mail.gq1.yahoo.com> In-Reply-To: <986041.3700.qm@web112005.mail.gq1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-hackers@FreeBSD.org, shrivatsan@gmail.com Subject: Re: sched_lock mutex and sleepq_wait X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 19:12:21 -0000 Shrivatsan wrote: > Hi, > > I am trying to understand how msleep() routine suspends the currently > executing thread. I see that msleep() calls sleepq_wait(). > > What I don't understand here is the way in which sched_lock mutex is > handled. > > I took the following snippet from FreeBSD 6: > > sleepq_wait(void *wchan) > { > MPASS(!(curthread->td_flags & TDF_SINTR)); > mtx_lock_spin(&sched_lock); > sleepq_switch(wchan); ----we sleep here----- some other thread runs, When it wakes up it magicall owns the schedlock which it releases. then it sleeps and locks schedlock again.A ----- we get run.. magically the schedlock is changed so that we own it. now we return, maybe 5 minutes later and must free it. > mtx_unlock_spin(&sched_lock); > } > > sched_lock mutex is held, and sleepq_switch() eventually calls > cpu_switch() which switches to a new thread. > > I don't exactly understand when the sched_lock mutex is released. > > Can someone please help me? > > Thanks, > -shrivatsan > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 23 21:47:49 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C76351065670; Tue, 23 Feb 2010 21:47:49 +0000 (UTC) (envelope-from eitanadlerlist@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 301158FC1C; Tue, 23 Feb 2010 21:47:48 +0000 (UTC) Received: by fxm23 with SMTP id 23so291924fxm.3 for ; Tue, 23 Feb 2010 13:47:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=ptEHvB5z2CKQ20+HL913OeO+HFB7CGHxnAVx/Wh9vQg=; b=RE8STmKr4DmkP20U85tRoVMrwKS/5a4EW6QuLmn6tnPTsZ7onVnoA+b5DdQbQprXqt h2w33k5RM/9cKsmyKDvZGkyvhGYMer1xrKcqdYndzzYH/5xI9iiPJojapv+VikdjGGav MUA0BDhZetD8pLkLBzwmH4gytkov9D4STO0MY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; b=w27XIhVEMChqIXmZQYNtShfqTOwLTXRrBqOTLJ7g7C4Dq07OiU4LbUsWaA4e+zylwF Ek1yzJGVi9CmBVTcVnEfb7eVga6Mx/Y+4GwVSHJyocpbJN1QYchS+L4G7VEbNYhArz7p iDqLGW6aZv50kV0MOA4Ts3ftAhkiE/URVqeRk= MIME-Version: 1.0 Received: by 10.239.187.139 with SMTP id l11mr670903hbh.96.1266961661781; Tue, 23 Feb 2010 13:47:41 -0800 (PST) In-Reply-To: References: <346566.86917.qm@web113519.mail.gq1.yahoo.com> <20100222120415.GA91890@mail.vnode.se> <16886.92863.qm@web113512.mail.gq1.yahoo.com> <4b82bf23.9713f30a.7dd8.0766@mx.google.com> From: Eitan Adler Date: Tue, 23 Feb 2010 23:47:21 +0200 Message-ID: To: Aditya Sarawgi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, "Pedro F. Giffuni" , joel@freebsd.org Subject: Re: Anyone updating the Project Ideas page? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 21:47:49 -0000 >> With gsoc just around the corner I think the project ideas page should >> be updated and moving the whole thing to wiki or maintaining a similar >> page on wiki(along with the original page) =A0sounds good to me. >> > I'll take the bait and make the page on the wiki. However someone else > will have to update the doc source to include a link.... > Bait done - I copied the projects, descriptions, links and other references= to http://wiki.freebsd.org/IdeasPage I also rewrote some small parts that make more sense now that it is on a wiki (from "see the wiki" to "see [[ThisPage]]) As it is a wiki feel free to update the page to make it look better, etc From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 23 22:00:04 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27BE3106566C; Tue, 23 Feb 2010 22:00:04 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id D33BA8FC17; Tue, 23 Feb 2010 22:00:03 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id BF1DD1E00127; Tue, 23 Feb 2010 23:00:02 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1NLoAcc067801; Tue, 23 Feb 2010 22:50:10 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1NLoAks067800; Tue, 23 Feb 2010 22:50:10 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Tue, 23 Feb 2010 22:50:10 +0100 To: freebsd-emulation@FreeBSD.org Message-ID: <20100223215010.GA67619@triton8.kn-bremen.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Tue, 23 Feb 2010 22:36:37 +0000 Cc: freebsd-hackers@FreeBSD.org, Tim Kientzle Subject: Linuxolator patches: stat and lseek SEEK_END for disk devices X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 22:00:04 -0000 Hi! Before this gets buried on -hackers in another thead... :) I now have disks appear as block devices for Linux processes (there already was commented out code for that in linux_stats.c, I hope my version is now `correct enough' to be usable [1]), and I made a simple patch to make lseek SEEK_END (L_XTND in the source) dtrt on disk devices too by simply invoking the DIOCGMEDIASIZE ioctl there; [2] both of these things are what (some) Linux processes expect. Patches are here: (made on stable/8, if they don't apply on head I'll have to make extra versions for that...) http://people.freebsd.org/~nox/linuxdisk-blk.patch [1] http://people.freebsd.org/~nox/lseek-seek_end.patch [2] And yes, with these patches the Linux bsdtar mentioned on -hackers in the `"tar tfv /dev/cd0" speedup patch' thread now also runs fast on FreeBSD. :) Cheers, Juergen From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 23 22:00:11 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 416631065670; Tue, 23 Feb 2010 22:00:11 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id D335A8FC16; Tue, 23 Feb 2010 22:00:03 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id D00821E0012B; Tue, 23 Feb 2010 23:00:02 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1NLrd4P067826; Tue, 23 Feb 2010 22:53:39 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1NLrdp2067825; Tue, 23 Feb 2010 22:53:39 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Tue, 23 Feb 2010 22:53:38 +0100 To: Tim Kientzle Message-ID: <20100223215338.GA67810@triton8.kn-bremen.de> References: <20100217215940.GA19713@triton8.kn-bremen.de> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> <201002200100.48161.jkim@FreeBSD.org> <20100220103329.GA35467@triton8.kn-bremen.de> <4B81DB31.70201@freebsd.org> <20100222230311.GA19535@triton8.kn-bremen.de> <4B835DFD.5060507@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B835DFD.5060507@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Tue, 23 Feb 2010 22:36:46 +0000 Cc: freebsd-hackers@freebsd.org, Juergen Lock Subject: Re: "tar tfv /dev/cd0" speedup patch; Linux compatibility patches X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 22:00:11 -0000 On Mon, Feb 22, 2010 at 08:47:57PM -0800, Tim Kientzle wrote: > Juergen Lock wrote: > > On Sun, Feb 21, 2010 at 05:17:37PM -0800, Tim Kientzle wrote: > >> > >> Could you try the current version of read_open_filename from: > >> > >> http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_filename.c > > > > Small but important bug: > > > > Index: archive_read_open_filename.c > > @@ -94,7 +94,7 @@ archive_read_open_filename(struct archiv > > struct read_file_data *mine; > > void *buffer; > > int fd; > > - int is_disk_like; > > + int is_disk_like = 0; > > Ah. Good catch! > > > Btw... the lseek + read combinations could also be replaced with > > pread(2), for the least amount of syscalls. :) > > Unfortunately, pread() isn't as portable. It's > an interesting idea, though; maybe I'll play with that. > Yeah this probably needs to be conditional on HAVE_PREAD etc. > > (Especially since I > > noticed this version at least when ran on an optical disc with iso9660 > > ends up doing lots of lseek()s towards the end without ever reading > > anything from the disc anymore...) > > Fortunately, because there are no reads, those lseek() > requests are pretty cheap. But you're right, it > wouldn't be hard to eliminate the unnecessary system > calls. > > > Patches are here: (made on stable/8, if they don't apply on head > > I'll have to make extra versions for that...) > > http://people.freebsd.org/~nox/linuxdisk-blk.patch [1] > > http://people.freebsd.org/~nox/lseek-seek_end.patch [2] > > I'll let someone more familiar with the linuxolator code > comment on those. I now made that an extra thread also on -emulation. Cheers, Juergen From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 12:03:26 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4A6C1065680 for ; Wed, 24 Feb 2010 12:03:26 +0000 (UTC) (envelope-from jacques.fourie@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 43D3F8FC1E for ; Wed, 24 Feb 2010 12:03:25 +0000 (UTC) Received: by wyb40 with SMTP id 40so1227385wyb.13 for ; Wed, 24 Feb 2010 04:03:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=YeprZ61WIx1wZTQf1aCXBLl+ZTlxbXqD1oCQnzGeE8A=; b=LVizGSp8RArWLGhnoCppOv5vUsQS+W5WyyKBeK4l7vJKl3+LUw4+vDaDWNOkNH+63R L7jsTW+DEa0yARSx2QAEAMjNDAqlg2yjUdO2/RGYx2uLVgMmTpL21/3IeJcF+EOS7jCo PnVoQbOy6KV7VUpnorz0oa5xpaxsmEjZ3Xeco= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=IOAzpwTFqpUTowVbJuYahUSSBjd6AbZMncXFXN9xnPAt9iU/bWbBmb9n5AAPdwrI57 EY6lm3WfDpnDUK6JdGoAPOnx7QmtE6DkXRuq78VVBLZC2mtrniiKJVO+E7BiEb60xFk1 OWbeJrIOOO9ac20U/ooXL1xzMYP+Err49fHWc= MIME-Version: 1.0 Received: by 10.216.88.148 with SMTP id a20mr518400wef.124.1267011691801; Wed, 24 Feb 2010 03:41:31 -0800 (PST) Date: Wed, 24 Feb 2010 13:41:31 +0200 Message-ID: From: Jacques Fourie To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: VirtualBox and vtophys X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 12:03:26 -0000 I've noticed that my virtualbox vboxdrv.ko started crashing after updating my 8.0-stable install. The crash occurs when vboxdrv calls vtophys() on a userland virtual address. Prior to r202894 this was working fine. Modifying pmap_kextract() to use vtopte() for non-kernel virtual adresses fixes the problem. Is it intended for vtophys() to still work on userland virtual addresses? From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 13:43:58 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BD791065670 for ; Wed, 24 Feb 2010 13:43:58 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout2.freenet.de (mout2.freenet.de [IPv6:2001:748:100:40::2:4]) by mx1.freebsd.org (Postfix) with ESMTP id AB09B8FC08 for ; Wed, 24 Feb 2010 13:43:57 +0000 (UTC) Received: from [195.4.92.19] (helo=9.mx.freenet.de) by mout2.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #1) id 1NkHWu-00040A-PX; Wed, 24 Feb 2010 14:43:56 +0100 Received: from p57ae25ad.dip0.t-ipconnect.de ([87.174.37.173]:23746 helo=ernst.jennejohn.org) by 9.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #2) id 1NkHWt-0008Ox-KO; Wed, 24 Feb 2010 14:43:55 +0100 Date: Wed, 24 Feb 2010 14:43:54 +0100 From: Gary Jennejohn To: Jacques Fourie Message-ID: <20100224144354.1283d7cd@ernst.jennejohn.org> In-Reply-To: References: X-Mailer: Claws Mail 3.7.4 (GTK+ 2.16.2; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: VirtualBox and vtophys X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 13:43:58 -0000 On Wed, 24 Feb 2010 13:41:31 +0200 Jacques Fourie wrote: > I've noticed that my virtualbox vboxdrv.ko started crashing after > updating my 8.0-stable install. The crash occurs when vboxdrv calls > vtophys() on a userland virtual address. Prior to r202894 this was > working fine. Modifying pmap_kextract() to use vtopte() for non-kernel > virtual adresses fixes the problem. Is it intended for vtophys() to > still work on userland virtual addresses? > This was recently (yesterday and today) discussed on freebsd-emulation with subject Virtualbox. See there for suggested actions. --- Gary Jennejohn From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 14:12:57 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FD2C1065670 for ; Wed, 24 Feb 2010 14:12:57 +0000 (UTC) (envelope-from andrey.zonov@gmail.com) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id 12FF38FC12 for ; Wed, 24 Feb 2010 14:12:56 +0000 (UTC) Received: by bwz8 with SMTP id 8so3709305bwz.3 for ; Wed, 24 Feb 2010 06:12:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=9/PvJmVBnMdVwJGbDaz4Rlm0a+Q10dgU/6QffKLy/2A=; b=iNPWJTtkvuAkJruBewr8JGnE+FXtSMDvdBAnPmHxLpyyl5cQ2s0OhpHJWWol7KmZjR 5m0B/wBv8hvPArsZQ47MDkt058POZ99nz+WMMrTBoyvi6ZWht0DzEWRJiW0L5cakf/uX ymE7dFAKvjD3T5/HwKAt/6k6+JVwWZBMemvVc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=BncLjU9TUDcd4P1nd9Sf8YcFk9Z2QIHMRd3nQRuOMVY6FLF4h7U8lZKBK7qDpAVCEQ Qtslt7ShG7gn5/xsyblbQ0rdyJcnkOXiRJhfBcdnMihYM0qF9E05eSV8WgwX35+DfpcS UL+vo4oH3MivvI6esbdorNye/0o+XQEmwkqNA= MIME-Version: 1.0 Received: by 10.204.36.70 with SMTP id s6mr306199bkd.22.1267019075357; Wed, 24 Feb 2010 05:44:35 -0800 (PST) Date: Wed, 24 Feb 2010 16:44:35 +0300 Message-ID: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> From: Andrey Zonov To: freebsd-hackers@freebsd.org Content-Type: multipart/mixed; boundary=000325554136e90b0b048058dddd Subject: 2 bytes allocated problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 14:12:57 -0000 --000325554136e90b0b048058dddd Content-Type: text/plain; charset=ISO-8859-1 Hi, When I try allocated pointer to a pointer, and in it some pointers (important: size is 2 bytes), the pointers lose their boundaries. Why it can happen? Test program in attach. PS in freebsd < 7, it's ok, in Linux too. -- Andrey Zonov --000325554136e90b0b048058dddd Content-Type: application/octet-stream; name="alloc.c" Content-Disposition: attachment; filename="alloc.c" Content-Transfer-Encoding: base64 X-Attachment-Id: f_g625qksd0 LyoKICovCgojaW5jbHVkZSA8ZXJyLmg+CiNpbmNsdWRlIDxzdGRpby5oPgojaW5jbHVkZSA8c3Rk bGliLmg+CiNpbmNsdWRlIDxzdHJpbmcuaD4KCiNkZWZpbmUgUzEgImFiIgojZGVmaW5lIFMyICJj ZCIKCnN0YXRpYyB2b2lkICpNYWxsb2Moc2l6ZV90IHNpemUpOwoKaW50Cm1haW4odm9pZCkKewoJ Y2hhciAqKnBwOwoKCXBwID0gKGNoYXIgKiopIE1hbGxvYygyICogc2l6ZW9mKGNoYXIgKikpOwoK CXBwWzBdID0gKGNoYXIgKikgbWFsbG9jKDIpOwoJbWVtY3B5KHBwWzBdLCBTMSwgMik7CglwcFsx XSA9IChjaGFyICopIG1hbGxvYygyKTsKCW1lbWNweShwcFsxXSwgUzIsIDIpOwoKCXByaW50Zigi JXNcbiIsICpwcCk7CglwcmludGYoIiVzXG4iLCBwcFswXSk7CglwcmludGYoIiVzXG4iLCBwcFsx XSk7CgoJZXhpdCgwKTsKfQoKc3RhdGljIHZvaWQgKgpNYWxsb2Moc2l6ZV90IHNpemUpCnsKCXZv aWQgKnA7CgoJcCA9IG1hbGxvYyhzaXplKTsKCWlmIChwID09IE5VTEwpCgkJZXJyKDEsICJtYWxs b2MoKSIpOwoKCXJldHVybihwKTsKfQo= --000325554136e90b0b048058dddd-- From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 14:25:16 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBA65106566C for ; Wed, 24 Feb 2010 14:25:16 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.17.10]) by mx1.freebsd.org (Postfix) with ESMTP id 820ED8FC1A for ; Wed, 24 Feb 2010 14:25:16 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-066-028-210.pools.arcor-ip.net [88.66.28.210]) by mrelayeu.kundenserver.de (node=mrbap0) with ESMTP (Nemesis) id 0MVb0p-1OGU9l06xT-00Ydaz; Wed, 24 Feb 2010 15:25:14 +0100 Received: (qmail 91410 invoked from network); 24 Feb 2010 14:25:13 -0000 Received: from f8x64.laiers.local (192.168.4.188) by laiers.local with SMTP; 24 Feb 2010 14:25:13 -0000 From: Max Laier Organization: FreeBSD To: freebsd-hackers@freebsd.org Date: Wed, 24 Feb 2010 15:25:11 +0100 User-Agent: KMail/1.12.4 (FreeBSD/8.0-RELEASE-p2; KDE/4.3.5; amd64; ; ) References: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> In-Reply-To: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201002241525.11930.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1/SD62JulL38UeQz6xv6hkyLr46RSwgZaNBQqs Ix5Di/LKvlr7y4AGJSgUjhqDeJ4bHAfdntmxtuq7m/b77fk3xG AYZkKju0QSmy8hEYyJDpQ== Cc: Andrey Zonov Subject: Re: 2 bytes allocated problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 14:25:17 -0000 On Wednesday 24 February 2010 14:44:35 Andrey Zonov wrote: > Hi, > > When I try allocated pointer to a pointer, and in it some pointers > (important: size is 2 bytes), the pointers lose their boundaries. > Why it can happen? > > Test program in attach. Your test program is broken: >#define S1 "ab" >#define S2 "cd" > > pp = (char **) Malloc(2 * sizeof(char *)); > > pp[0] = (char *) malloc(2); > memcpy(pp[0], S1, 2); > pp[1] = (char *) malloc(2); > memcpy(pp[1], S2, 2); > > printf("%s\n", *pp); > printf("%s\n", pp[0]); > printf("%s\n", pp[1]); Why should *pp == pp[0], or pp[1] be a nul-terminated string? You just copied the two characters. It's pure luck if there is a \0 at the end of any of these elements, or that the access doesn't cause a SEGV. If you do: > pp[0] = (char *) malloc(3); > memcpy(pp[0], S1, 3); > pp[1] = (char *) malloc(3); > memcpy(pp[1], S2, 3); instead, you copy the termination and things work as expected. Regards, Max From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 14:35:46 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C9421065672 for ; Wed, 24 Feb 2010 14:35:46 +0000 (UTC) (envelope-from giovanni.trematerra@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id A74A68FC0C for ; Wed, 24 Feb 2010 14:35:45 +0000 (UTC) Received: by fxm23 with SMTP id 23so1070743fxm.3 for ; Wed, 24 Feb 2010 06:35:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=LdI23FcfAcBM4svdLbOsRCWNXCN+tuUXrt9eMrpjrzU=; b=HbhLKKRN8NBrsDIqlqIb98KkYIi86bsADvl8dVAAPUq+brgbq3snu2sSHD2flmakH+ IZaO20D7Ym/thwaHjOlW2xy9f7qE5CTWMdMEWc06AbWn9lu/zb2/FxxuRAzdS1XQOsNX hho05eC753Up1XYX1lvnyp09enz73X07uQNb0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Gzv3IBSFJYRfOD1MDpVQcHAGF1aKJ6hY4Cw9iCMTxxZh5sLHDRogwufqs+HpP61V4X 8gHZthic7cbDArzOLT9OLP/GBgwaKaq9vzYUb/p6/YN5IiU2C0PiIKOV7NriRF5Wh9CA 2kqTFzYJccTiAiXUn3cKIWH5gq5mdkJCSA0ck= MIME-Version: 1.0 Received: by 10.223.77.139 with SMTP id g11mr2932580fak.4.1267020321560; Wed, 24 Feb 2010 06:05:21 -0800 (PST) In-Reply-To: References: Date: Wed, 24 Feb 2010 15:05:21 +0100 Message-ID: <4e6cba831002240605g17cd3ed7l60b635cedba2ae77@mail.gmail.com> From: Giovanni Trematerra To: Jacques Fourie Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: VirtualBox and vtophys X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 14:35:46 -0000 On Wed, Feb 24, 2010 at 12:41 PM, Jacques Fourie wrote: > I've noticed that my virtualbox vboxdrv.ko started crashing after > updating my 8.0-stable install. The crash occurs when vboxdrv calls > vtophys() on a userland virtual address. Prior to r202894 this was > working fine. Modifying pmap_kextract() to use vtopte() for non-kernel > virtual adresses fixes the problem. Is it intended for vtophys() to > still work on userland virtual addresses? No, vtophys is not intended to be used along with user virtual addresses. -- Gianni From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 14:56:13 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11E691065673 for ; Wed, 24 Feb 2010 14:56:13 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id DA8D58FC14 for ; Wed, 24 Feb 2010 14:56:12 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 9747F46B53; Wed, 24 Feb 2010 09:56:12 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id DDC3B8A024; Wed, 24 Feb 2010 09:56:11 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Wed, 24 Feb 2010 09:43:20 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20100120; KDE/4.3.1; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002240943.20648.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 24 Feb 2010 09:56:11 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.3 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Jacques Fourie Subject: Re: VirtualBox and vtophys X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 14:56:13 -0000 On Wednesday 24 February 2010 6:41:31 am Jacques Fourie wrote: > I've noticed that my virtualbox vboxdrv.ko started crashing after > updating my 8.0-stable install. The crash occurs when vboxdrv calls > vtophys() on a userland virtual address. Prior to r202894 this was > working fine. Modifying pmap_kextract() to use vtopte() for non-kernel > virtual adresses fixes the problem. Is it intended for vtophys() to > still work on userland virtual addresses? Use pmap_extract(pmap, ...) for user addresses. Granted, that is only reliable if the user pages have been wired in some fashion. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 15:14:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15A3B1065672 for ; Wed, 24 Feb 2010 15:14:59 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id CCE818FC1F for ; Wed, 24 Feb 2010 15:14:58 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id AC1671FFC22; Wed, 24 Feb 2010 15:14:56 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 3D11F844EE; Wed, 24 Feb 2010 16:14:56 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Andrey Zonov References: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> Date: Wed, 24 Feb 2010 16:14:56 +0100 In-Reply-To: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> (Andrey Zonov's message of "Wed, 24 Feb 2010 16:44:35 +0300") Message-ID: <86eikar7gv.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: 2 bytes allocated problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 15:14:59 -0000 Andrey Zonov writes: > When I try allocated pointer to a pointer, and in it some pointers > (important: size is 2 bytes), the pointers lose their boundaries. Pointers have no boundareis in C. > PS in freebsd < 7, it's ok, in Linux too. Only by accident. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 15:32:22 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BB81106564A; Wed, 24 Feb 2010 15:32:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 464538FC13; Wed, 24 Feb 2010 15:32:20 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1OFWEtU010589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Feb 2010 17:32:14 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o1OFWD6O091278; Wed, 24 Feb 2010 17:32:13 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o1OFWDNO091277; Wed, 24 Feb 2010 17:32:13 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 24 Feb 2010 17:32:13 +0200 From: Kostik Belousov To: John Baldwin Message-ID: <20100224153213.GV50403@deviant.kiev.zoral.com.ua> References: <201002240943.20648.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="SLRDCdsRrCd4mzuj" Content-Disposition: inline In-Reply-To: <201002240943.20648.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, Jacques Fourie Subject: Re: VirtualBox and vtophys X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 15:32:22 -0000 --SLRDCdsRrCd4mzuj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 24, 2010 at 09:43:20AM -0500, John Baldwin wrote: > On Wednesday 24 February 2010 6:41:31 am Jacques Fourie wrote: > > I've noticed that my virtualbox vboxdrv.ko started crashing after > > updating my 8.0-stable install. The crash occurs when vboxdrv calls > > vtophys() on a userland virtual address. Prior to r202894 this was > > working fine. Modifying pmap_kextract() to use vtopte() for non-kernel > > virtual adresses fixes the problem. Is it intended for vtophys() to > > still work on userland virtual addresses? >=20 > Use pmap_extract(pmap, ...) for user addresses. Granted, that is only=20 > reliable if the user pages have been wired in some fashion. The reference code is in sys/dev/cxgb/ulp/tom/cxgb_vm.c:vm_fault_hold_user_pages(), it holds and returns the page that was mapped at the moment of the call at the specified address. --SLRDCdsRrCd4mzuj Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkuFRn0ACgkQC3+MBN1Mb4i3cwCaAgF2YabOhA6lf52wBRWxhq/q ytsAniAED5NSA0BOrK2A4OMYMBDlR3D6 =Ngfp -----END PGP SIGNATURE----- --SLRDCdsRrCd4mzuj-- From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 18:17:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E12DA106564A for ; Wed, 24 Feb 2010 18:17:53 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505e.appriver.com [98.129.35.9]) by mx1.freebsd.org (Postfix) with ESMTP id A77C28FC0A for ; Wed, 24 Feb 2010 18:17:53 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.45 X-Note-Reverse-DNS: ht03.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.45] (HELO HT03.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 25472348 for freebsd-hackers@freebsd.org; Wed, 24 Feb 2010 12:17:52 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by HT03.exg5.exghost.com ([10.242.228.75]) with mapi; Wed, 24 Feb 2010 12:17:51 -0600 From: Peter Steele To: "freebsd-hackers@freebsd.org" Date: Wed, 24 Feb 2010 12:17:50 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acqz1wAQdYoZ2dInQG2e7o15W4OUdwAAMdUwAGMe0qA= Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 18:17:54 -0000 >You're going to need a debug version of libc, too. gdb won't be able to f= ind a backtrace out of a libc function without it. What's the proper way to build a debug version of libc and the other librar= ies? I tried this: export CFLAGS=3D"-O0" make buildworld make installworld DESTDIR=3D/mydir and then copied libc.so.7 from /mydir/lib to the /lib dir on my target syst= em. I also replaced the ntpd binary with the debug version. I can see that = -O0 is being used in the various "cc" commands that are generated, but libc= still doesn't seem to be built properly. When I attach to a hung ntpd proc= ess, I get this: # gdb /usr/sbin/ntpd -p 2113 GNU gdb 6.1.1 [FreeBSD] ... Attaching to program: /usr/sbin/ntpd, process 2113 Reading symbols from /lib/libm.so.5...(no debugging symbols found)...done. ... Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done= . ... [Switching to Thread 8012041c0 (LWP 100283)] 0x0000000800dbeddc in select () from /lib/libc.so.7 (gdb) bt #0 0x0000000800dbeddc in select () from /lib/libc.so.7 #1 0x00000000004335de in ntpdmain () #2 0x000000000043310b in main () So I'm getting some symbols from ntpd but I still can't see into select(). = It hangs in there forever so that's where I need to drill down further. How= do I get libc built with full debug symbols? In other testing I've narrowed the problem down to some kind of Python issu= e. If I run the Python code at the end of this email where "ntpd -g -q" is = launched as part of a Python thread class, the command hangs (the code assu= mes that ntpd is not already running). If I run the same ntpd command in a = normal function (e.g. main) no hang occurs. I've tried subcommand.Popen and= os.spawnv to run ntpd and these calls behave exactly the same way--when ca= lled from a thread the ntpd process hangs but it works fine when called fro= m outside of a thread. This is a breakdown of course of our larger project = into a simple test app. In our real code we cannot so easily eliminate this= thread wrapper. The same code BTW works fine on our FreeBSD 7 boxes, the main difference be= ing we are running an older version of Python on those boxes (2.5.1 instead= of 2.6.2). I tried installing the same 2.5.1 package on a FBSD 8 box and t= hat solved the problem. Curiously a slightly newer FBSD 7 version of Python= , 2.5.5, causes the same hang to occur. So only Python 2.5.1 built under Fr= eeBSD 7 works to get around this issue with ntpd on FreeBSD 8. That means o= ne potential solution is to downgrade to this 2.5.1, but we have other libr= aries targeted to work with Python 2.6 and we don't really want to downgrad= e all these associated libraries. If anyone has any clues at all as to what is causing this issue, I'd apprec= iate the feedback. Here's the code that reproduces this behavior. #! /usr/bin/env python import os import threading class RunProc(threading.Thread): def __init__(self, cmd): threading.Thread.__init__(self) self.cmd =3D cmd def run(self): os.system(self.cmd) def main(): RunProc("/usr/sbin/ntpd -g -q").start() if __name__ =3D=3D "__main__": main() From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 19:00:47 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3960F1065670 for ; Wed, 24 Feb 2010 19:00:47 +0000 (UTC) (envelope-from shuvaev@physik.uni-wuerzburg.de) Received: from mailrelay.rz.uni-wuerzburg.de (mailrelay.rz.uni-wuerzburg.de [132.187.3.28]) by mx1.freebsd.org (Postfix) with ESMTP id B6F908FC17 for ; Wed, 24 Feb 2010 19:00:46 +0000 (UTC) Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 56B135AC5F; Wed, 24 Feb 2010 20:00:37 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 533D85AC4D; Wed, 24 Feb 2010 20:00:37 +0100 (CET) X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de Received: from mail.physik.uni-wuerzburg.de (wthp192.physik.uni-wuerzburg.de [132.187.40.192]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 315E25D7B1; Wed, 24 Feb 2010 20:00:37 +0100 (CET) Received: from wep4035 ([132.187.37.35]) by mail.physik.uni-wuerzburg.de (Lotus Domino Release 8.5.1FP1) with ESMTP id 2010022420003564-53182 ; Wed, 24 Feb 2010 20:00:35 +0100 Received: by wep4035 (sSMTP sendmail emulation); Wed, 24 Feb 2010 20:00:35 +0100 Date: Wed, 24 Feb 2010 20:00:35 +0100 From: Alexey Shuvaev To: Peter Steele Message-ID: <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> MIME-Version: 1.0 In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> Organization: Universitaet Wuerzburg User-Agent: Mutt/1.5.20 (2009-06-14) X-MIMETrack: Itemize by SMTP Server on domino1/uni-wuerzburg(Release 8.5.1FP1|January 05, 2010) at 02/24/2010 08:00:36 PM, Serialize by Router on domino1/uni-wuerzburg(Release 8.5.1FP1|January 05, 2010) at 02/24/2010 08:00:36 PM, Serialize complete at 02/24/2010 08:00:36 PM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Cc: "freebsd-hackers@freebsd.org" Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 19:00:47 -0000 On Wed, Feb 24, 2010 at 12:17:50PM -0600, Peter Steele wrote: > >You're going to need a debug version of libc, too. > >gdb won't be able to find a backtrace out of a libc function without it. > > What's the proper way to build a debug version of libc and the other > libraries? I tried this: > > export CFLAGS="-O0" > make buildworld > make installworld DESTDIR=/mydir > > and then copied libc.so.7 from /mydir/lib to the /lib dir on > my target system. I also replaced the ntpd binary with the debug version. > I can see that -O0 is being used in the various "cc" commands that are > generated, but libc still doesn't seem to be built properly. > When I attach to a hung ntpd process, I get this: > > # gdb /usr/sbin/ntpd -p 2113 > GNU gdb 6.1.1 [FreeBSD] > ... > Attaching to program: /usr/sbin/ntpd, process 2113 > Reading symbols from /lib/libm.so.5...(no debugging symbols found)...done. > ... > Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. > Loaded symbols for /lib/libc.so.7 > Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done. > ... > [Switching to Thread 8012041c0 (LWP 100283)] > 0x0000000800dbeddc in select () from /lib/libc.so.7 > (gdb) bt > #0 0x0000000800dbeddc in select () from /lib/libc.so.7 > #1 0x00000000004335de in ntpdmain () > #2 0x000000000043310b in main () > > So I'm getting some symbols from ntpd but I still can't see into select(). > I think not. > It hangs in there forever so that's where I need to drill down further. > How do I get libc built with full debug symbols? > I haven't tried it by myself but think here is the way to go: put the following to /etc/make.conf and recompile needed libraries / ports. WITH_DEBUG=yes DEBUG_FLAGS=-g This should do the trick for both base and ports. > > [snip] > > If anyone has any clues at all as to what is causing this issue, > I'd appreciate the feedback. Here's the code that reproduces this behavior. > > #! /usr/bin/env python > import os > import threading > > class RunProc(threading.Thread): > def __init__(self, cmd): > threading.Thread.__init__(self) > self.cmd = cmd > > def run(self): > os.system(self.cmd) > > def main(): > RunProc("/usr/sbin/ntpd -g -q").start() > > if __name__ == "__main__": > main() > Mmm... Do other daemons (sshd, lpd, ...) also fail when started through this script? Normal commands (ls, ps) seem not affected. 0.02$, Alexey. From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 19:37:57 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FD36106564A for ; Wed, 24 Feb 2010 19:37:57 +0000 (UTC) (envelope-from andrey.zonov@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id BE8688FC14 for ; Wed, 24 Feb 2010 19:37:56 +0000 (UTC) Received: by fxm23 with SMTP id 23so14533fxm.3 for ; Wed, 24 Feb 2010 11:37:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=fnsWtVflDrXhOdSKwp8SFJ4701NLumVQzc8C7cY+AJQ=; b=DOsS7ciuiKluFC+RrlRBrJ4MUg/suUivQIYLUNC0Y55oYHLJxpH+y/BHSyA8j/dM4z GSb5WG1LbHue/kpuQD8BRCIiFRjYLzJyz4NqyxsvjUFBMysb1vE3GwLkMyW/G1V96zDL mTwymNtKcdu03L6ZsP1LIn82xWdqvXD9xstq8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=Q6LUvm9shzUauA3qb1h7CWEQWPuUzuLKZXNZMI/M1IMFXQrdyDjkKxgDnjl1QlJfuI TRtufaILTInR8GSbW2ql4hbu+aelqfZJqmgDKl8phq5+zIpNA7ExtNyR5Y9sRR7UQcNX I+MY8bhk3VFW+8YZbTP9CMwS6YlYR0jZbTpfg= Received: by 10.87.46.12 with SMTP id y12mr706497fgj.47.1267040272070; Wed, 24 Feb 2010 11:37:52 -0800 (PST) Received: from ?10.254.254.77? (ppp79-139-206-202.pppoe.spdop.ru [79.139.206.202]) by mx.google.com with ESMTPS id 13sm3680274fxm.10.2010.02.24.11.37.50 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 24 Feb 2010 11:37:50 -0800 (PST) Message-ID: <4B858007.1000008@gmail.com> Date: Wed, 24 Feb 2010 22:37:43 +0300 From: Andrey Zonov User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> <86eikar7gv.fsf@ds4.des.no> In-Reply-To: <86eikar7gv.fsf@ds4.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org Subject: Re: 2 bytes allocated problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 19:37:57 -0000 And how free() finds that the need to release? Dag-Erling Smørgrav пишет: > Andrey Zonov writes: > >> When I try allocated pointer to a pointer, and in it some pointers >> (important: size is 2 bytes), the pointers lose their boundaries. >> > > Pointers have no boundareis in C. > > >> PS in freebsd < 7, it's ok, in Linux too. >> > > Only by accident. > > DES > From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 19:55:16 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85D28106566B for ; Wed, 24 Feb 2010 19:55:16 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 471368FC0A for ; Wed, 24 Feb 2010 19:55:16 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 4761C1FFC4D; Wed, 24 Feb 2010 19:55:15 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 28032844B0; Wed, 24 Feb 2010 20:55:15 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Andrey Zonov References: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> <86eikar7gv.fsf@ds4.des.no> <4B858007.1000008@gmail.com> Date: Wed, 24 Feb 2010 20:55:15 +0100 In-Reply-To: <4B858007.1000008@gmail.com> (Andrey Zonov's message of "Wed, 24 Feb 2010 22:37:43 +0300") Message-ID: <86sk8qpfx8.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: 2 bytes allocated problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 19:55:16 -0000 Andrey Zonov writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Pointers have no boundareis in C. > And how free() finds that the need to release? That is a very simple question with a very complicated answer. Whole books have been written about the subject. Normally, I'd say "look it up on Wikipedia", but the Wikipedia article on dynamic memory allocation is little more than a stub. Try Knuth's The Art of Computer Programming instead. However, none of this changes the fact that pointers in C have no boundaries. In practical terms, a pointer is just a number that refers to a particular location in memory. If you do char *p =3D malloc(10); strcpy(p, "abcdefghi") p +=3D 5; then *p =3D=3D p[0] =3D=3D 'f', and if printf("%s", p) will print "fghi". = What happens if you then try to free(p) will vary from OS to OS and sometimes between versions of the same OS; in most cases, either nothing will happen at all, or your program will crash. The reason printf() knows to stop after the 'i' is that the next char in memory is 0. That's why your program didn't work: there was no 0 there to indicate the end of the string. Sometimes it would seem to work because there would, by coincidence, be a 0 there already, but that doesn't mean your code is correct. Why is there a 0 after the 'i'? Because when you write "abcdefghi", the compiler actually stores "abcdefghi\0". That's the definition of "string" in C: a sequence of characters immediately followed by a 0. If you don't want the 0 there, you have to do something like this: char a[9] =3D { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i' }; but then you don't have a string, just an array of 9 chars. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 20:37:10 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D0A6106566B for ; Wed, 24 Feb 2010 20:37:10 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-qy0-f193.google.com (mail-qy0-f193.google.com [209.85.221.193]) by mx1.freebsd.org (Postfix) with ESMTP id CD9E48FC1C for ; Wed, 24 Feb 2010 20:37:09 +0000 (UTC) Received: by qyk31 with SMTP id 31so570520qyk.27 for ; Wed, 24 Feb 2010 12:37:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:subject:content-type; bh=J0IpYUS8S6kNBEvR+gM0wGncz+Y8O8fsrOfcSsLwqMY=; b=cJTMII4M7TSZsylxuHRnshPXGBdWuIkrmmfuf4jYoN+rHvtHbUwUq98hYIMnZUIVkF 99Zsis9ALijLoBqIUB+ezlimoWEtsQGRB9xJn7n4LjGb+yXh4Foz/VnPPcRckYetvTCI 1ZD68WDnLvFN8rZFNkjAPNcBOiGSTZhOK7K3g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type; b=ZiVPH35QAKTEM7SbG7hshKI0hQe7c/xnVMbV+BUvuEYKvcZt4kwNCQj+LOFhMt4yOv nwqFHKF9rmhy1gIher5ipILfl/N3XJuX9C8KfoOzVDYGIQxC7gKG5bfWcP5ZilQOGuJI 8Q6ijZ0EAxhe6gyTUkAsLdLweYS3N1LOkt5Ok= Received: by 10.224.96.20 with SMTP id f20mr259990qan.145.1267043819017; Wed, 24 Feb 2010 12:36:59 -0800 (PST) Received: from aryeh-desktop.istudentunion.com (ool-44c0cd7a.dyn.optonline.net [68.192.205.122]) by mx.google.com with ESMTPS id 20sm607800qyk.12.2010.02.24.12.36.57 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 24 Feb 2010 12:36:58 -0800 (PST) Message-ID: <4B858DEB.6090509@gmail.com> Date: Wed, 24 Feb 2010 15:36:59 -0500 From: "Aryeh M. Friedman" User-Agent: Thunderbird 2.0.0.23 (X11/20100120) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: multipart/mixed; boundary="------------070700010900020904090004" Subject: [Fwd: odd nfs behaviour] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 20:37:10 -0000 This is a multi-part message in MIME format. --------------070700010900020904090004 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit --------------070700010900020904090004 Content-Type: message/rfc822; name="odd nfs behaviour.eml" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="odd nfs behaviour.eml" X-Mozilla-Keys: Message-ID: <4B85881B.2060100@gmail.com> Date: Wed, 24 Feb 2010 15:12:11 -0500 From: "Aryeh M. Friedman" User-Agent: Thunderbird 2.0.0.23 (X11/20100120) MIME-Version: 1.0 To: freebas-hackers@freebsd.org Subject: odd nfs behaviour Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit I have home dir's nfs mounted via amd on the jack the following error occurred on (I do not have NIS I do uid/gid syncing manually): ~/master.C091 aryeh@kate% telnet jack Trying 72.89.191.146... Connected to jack.istudentunion.com. Escape character is '^]'. Trying SRA secure login: User (aryeh): aryeh Password: [ SRA accepts you ] FreeBSD/i386 (jack.istudentunion.com) (pts/0) nfs server pid52@jack:/host: not responding nfs server pid52@jack:/host: not responding nfs server pid52@jack:/host: not responding nfs server pid52@jack:/host: not responding nfs server pid52@jack:/host: not responding nfs server pid52@jack:/host: not responding NFS Server: FreeBSD kate.istudentunion.com 7.2-RELEASE-p1 FreeBSD 7.2-RELEASE-p1 #0: Tue Jun 9 21:30:43 UTC 2009 root@i386-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386 NFS Client: FreeBSD jack.istudentunion.com 8.0-STABLE FreeBSD 8.0-STABLE #0: Thu Jan 7 02:15:16 UTC 2010 root@:/usr/obj/usr/src/sys/GENERIC i386 --------------070700010900020904090004-- From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 21:31:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57363106566B for ; Wed, 24 Feb 2010 21:31:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 27C3D8FC0C for ; Wed, 24 Feb 2010 21:31:36 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id B8D3D46B53; Wed, 24 Feb 2010 16:31:35 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id F0DB48A01F; Wed, 24 Feb 2010 16:31:34 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Wed, 24 Feb 2010 15:17:25 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002241517.25659.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 24 Feb 2010 16:31:34 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.4 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Peter Steele Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 21:31:36 -0000 On Wednesday 24 February 2010 1:17:50 pm Peter Steele wrote: > >You're going to need a debug version of libc, too. gdb won't be able to find a backtrace out of a libc function without it. > > What's the proper way to build a debug version of libc and the other libraries? I tried this: You can just do this: cd /usr/src/lib/libc make clean make DEBUG_FLAGS=-g make install -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 21:50:48 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2ABD6106564A for ; Wed, 24 Feb 2010 21:50:48 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505c.appriver.com [98.129.35.7]) by mx1.freebsd.org (Postfix) with ESMTP id E78A38FC2B for ; Wed, 24 Feb 2010 21:50:47 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.15 X-Note-Reverse-DNS: ht02.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.15] (HELO ht02.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 29424976 for freebsd-hackers@freebsd.org; Wed, 24 Feb 2010 15:50:47 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by ht02.exg5.exghost.com ([98.129.23.15]) with mapi; Wed, 24 Feb 2010 15:50:46 -0600 From: Peter Steele To: "freebsd-hackers@freebsd.org" Date: Wed, 24 Feb 2010 15:50:45 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acq1mMHRykm89mAyR6WUmQ3QkA228gAAjIAw Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95519@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <201002241517.25659.jhb@freebsd.org> In-Reply-To: <201002241517.25659.jhb@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 21:50:48 -0000 >> What's the proper way to build a debug version of libc and the other lib= raries? I tried this: > >You can just do this: > >cd /usr/src/lib/libc >make clean >make DEBUG_FLAGS=3D-g >make install When I tried this the make actually failed with various errors. So I decide= d to do a full "make buildworld DEBUG_FLAGS=3D-g" but in looking at the out= put being generated I see see -O2 in the cc commands and this at least shou= ld be -O0. It doesn't look like the DEBUG_FLAGS is having any effect. From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 21:56:38 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BAC91065675 for ; Wed, 24 Feb 2010 21:56:38 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505c.appriver.com [98.129.35.7]) by mx1.freebsd.org (Postfix) with ESMTP id C59798FC17 for ; Wed, 24 Feb 2010 21:56:37 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.15 X-Note-Reverse-DNS: ht02.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.15] (HELO ht02.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 29426402 for freebsd-hackers@freebsd.org; Wed, 24 Feb 2010 15:56:36 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by ht02.exg5.exghost.com ([98.129.23.15]) with mapi; Wed, 24 Feb 2010 15:56:36 -0600 From: Peter Steele To: "freebsd-hackers@freebsd.org" Date: Wed, 24 Feb 2010 15:56:35 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acq1g6q9iX5Mk83lSJ2rLhjxnsGOLwAF8vgw Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> In-Reply-To: <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 21:56:38 -0000 >> How do I get libc built with full debug symbols? >=20 >I haven't tried it by myself but think here is the way to go: put the foll= owing to /etc/make.conf and recompile needed libraries / ports. >WITH_DEBUG=3Dyes >DEBUG_FLAGS=3D-g That didn't seem to have any effect. I still see -O2 being used instead of = -O0. >Mmm... Do other daemons (sshd, lpd, ...) also fail when started through th= is script? Normal commands (ls, ps) seem not affected. I tried a few other things and they all seemed to run correctly. We use thi= s same general approach in the full version of this script to launch lots o= f applications. Its role in fact is a process launcher/monitor. I stripped = it down to the bare minimum in order to isolate the cause of the problem. I= t seems that only ntpd hangs, but not if I use Python 2.5.1. From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 22:09:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA77F1065672; Wed, 24 Feb 2010 22:09:54 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 25E0B8FC14; Wed, 24 Feb 2010 22:09:53 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1OM9loF041157 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Feb 2010 00:09:47 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o1OM9l6d093930; Thu, 25 Feb 2010 00:09:47 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o1OM9lBq093929; Thu, 25 Feb 2010 00:09:47 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 25 Feb 2010 00:09:47 +0200 From: Kostik Belousov To: John Baldwin Message-ID: <20100224220947.GX50403@deviant.kiev.zoral.com.ua> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <201002241517.25659.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="M2XkPJ45YXpBrBm3" Content-Disposition: inline In-Reply-To: <201002241517.25659.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, Peter Steele Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 22:09:54 -0000 --M2XkPJ45YXpBrBm3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 24, 2010 at 03:17:25PM -0500, John Baldwin wrote: > On Wednesday 24 February 2010 1:17:50 pm Peter Steele wrote: > > >You're going to need a debug version of libc, too. gdb won't be able = to=20 > find a backtrace out of a libc function without it. > >=20 > > What's the proper way to build a debug version of libc and the other=20 > libraries? I tried this: >=20 > You can just do this: >=20 > cd /usr/src/lib/libc > make clean > make DEBUG_FLAGS=3D-g > make install make install should be done with DEBUG_FLAGS containing -g too, otherwise strip(1) is called on the installed binary. --M2XkPJ45YXpBrBm3 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkuFo6sACgkQC3+MBN1Mb4iXbQCgvRxovauBC3d6Dr+qk+YmF1Wu /KIAoMFR3HcgAhLrtAYmQHWcHBd8z34z =zNx8 -----END PGP SIGNATURE----- --M2XkPJ45YXpBrBm3-- From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 22:18:44 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 307DE1065672 for ; Wed, 24 Feb 2010 22:18:44 +0000 (UTC) (envelope-from nate@thatsmathematics.com) Received: from euclid.ucsd.edu (euclid.ucsd.edu [132.239.145.52]) by mx1.freebsd.org (Postfix) with ESMTP id 108058FC08 for ; Wed, 24 Feb 2010 22:18:43 +0000 (UTC) Received: from zeno.ucsd.edu (zeno.ucsd.edu [132.239.145.22]) by euclid.ucsd.edu (8.11.7p3+Sun/8.11.7) with ESMTP id o1OMIhY29893; Wed, 24 Feb 2010 14:18:43 -0800 (PST) Received: from localhost (neldredg@localhost) by zeno.ucsd.edu (8.11.7p3+Sun/8.11.7) with ESMTP id o1OMIh023946; Wed, 24 Feb 2010 14:18:43 -0800 (PST) X-Authentication-Warning: zeno.ucsd.edu: neldredg owned process doing -bs Date: Wed, 24 Feb 2010 14:18:42 -0800 (PST) From: Nate Eldredge X-X-Sender: neldredg@zeno.ucsd.edu To: Peter Steele In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> Message-ID: References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: "freebsd-hackers@freebsd.org" Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 22:18:44 -0000 On Mon, 22 Feb 2010, Peter Steele wrote: >> Just out of curiosity, can you attach to the process via gdb and get a >> backtrace? This smells like a locked pthread_join I hit in my own code >> a few weeks ago > > I'm not using the debug version of ntpd so the backtrace isn't too > useful, but here's what I get: > > (gdb) bt > #0 0x0000000800d52bfc in select () from /lib/libc.so.7 > #1 0x0000000000425273 in ?? () > #2 0x000000000040540e in ?? () > #3 0x0000000800580000 in ?? () > #4 0x0000000000000000 in ?? () I bet ntpd doesn't call select() in all that many places. Instead of going to all this trouble to build a debugging libc, you could just grep for select() and place breakpoints on all occurrences. (It might also be obvious from looking at them which one is the offender.) Also, since a system call is causing the trouble, you might learn something from truss or ktrace. -- Nate Eldredge nate@thatsmathematics.com From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 22:36:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60557106566C for ; Wed, 24 Feb 2010 22:36:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 31FDC8FC18 for ; Wed, 24 Feb 2010 22:36:35 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id AF32046B06; Wed, 24 Feb 2010 17:36:34 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 4DF388A01F; Wed, 24 Feb 2010 17:36:22 -0500 (EST) From: John Baldwin To: Kostik Belousov Date: Wed, 24 Feb 2010 17:24:40 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <201002241517.25659.jhb@freebsd.org> <20100224220947.GX50403@deviant.kiev.zoral.com.ua> In-Reply-To: <20100224220947.GX50403@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201002241724.40876.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 24 Feb 2010 17:36:22 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.4 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-hackers@freebsd.org, Peter Steele Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 22:36:35 -0000 On Wednesday 24 February 2010 5:09:47 pm Kostik Belousov wrote: > On Wed, Feb 24, 2010 at 03:17:25PM -0500, John Baldwin wrote: > > On Wednesday 24 February 2010 1:17:50 pm Peter Steele wrote: > > > >You're going to need a debug version of libc, too. gdb won't be able to > > find a backtrace out of a libc function without it. > > > > > > What's the proper way to build a debug version of libc and the other > > libraries? I tried this: > > > > You can just do this: > > > > cd /usr/src/lib/libc > > make clean > > make DEBUG_FLAGS=-g > > make install > > make install should be done with DEBUG_FLAGS containing -g too, otherwise > strip(1) is called on the installed binary. Doh, yes. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 22:39:33 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C17C1065670 for ; Wed, 24 Feb 2010 22:39:33 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505a.appriver.com [98.129.35.4]) by mx1.freebsd.org (Postfix) with ESMTP id 5444F8FC1F for ; Wed, 24 Feb 2010 22:39:33 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.14 X-Note-Reverse-DNS: ht01.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.14] (HELO ht01.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 29436160 for freebsd-hackers@freebsd.org; Wed, 24 Feb 2010 16:39:32 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by ht01.exg5.exghost.com ([98.129.23.14]) with mapi; Wed, 24 Feb 2010 16:39:32 -0600 From: Peter Steele To: "freebsd-hackers@freebsd.org" Date: Wed, 24 Feb 2010 16:39:31 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acq1n1syDMzQU1d0TDWH/1XhPciVzAAATeaA Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95567@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 22:39:33 -0000 >I bet ntpd doesn't call select() in all that many places. Instead of goin= g to all this trouble to build a debugging libc, you could just >grep for select() and place breakpoints on all occurrences. (It might als= o be obvious from looking at them which one is the offender.) I just checked--there are five calls to select. I might flag each one with = a printf or something and recompile to see which one is the culprit. >Also, since a system call is causing the trouble, you might learn somethin= g from truss or ktrace. I'll check these out... From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 22:40:50 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE4C51065688 for ; Wed, 24 Feb 2010 22:40:50 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505a.appriver.com [98.129.35.4]) by mx1.freebsd.org (Postfix) with ESMTP id 81E348FC2E for ; Wed, 24 Feb 2010 22:40:50 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Policy: GLOBAL - maxiscale.com X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.45 X-Note-Reverse-DNS: ht03.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.45] (HELO HT03.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 29436428; Wed, 24 Feb 2010 16:40:48 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by HT03.exg5.exghost.com ([10.242.228.75]) with mapi; Wed, 24 Feb 2010 16:40:48 -0600 From: Peter Steele To: John Baldwin , Kostik Belousov Date: Wed, 24 Feb 2010 16:40:47 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acq1odVHbFmZpXEQSOWN18WrA3zH+QAAH0HQ Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95569@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <201002241517.25659.jhb@freebsd.org> <20100224220947.GX50403@deviant.kiev.zoral.com.ua> <201002241724.40876.jhb@freebsd.org> In-Reply-To: <201002241724.40876.jhb@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-hackers@freebsd.org" Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 22:40:50 -0000 >> make install should be done with DEBUG_FLAGS containing -g too, otherwis= e >> strip(1) is called on the installed binary. > >Doh, yes. I did not do this; that's likely my problem. Thanks. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 08:30:37 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02561106567B for ; Thu, 25 Feb 2010 08:30:37 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 80E088FC16 for ; Thu, 25 Feb 2010 08:30:36 +0000 (UTC) Received: by wyb40 with SMTP id 40so1888177wyb.13 for ; Thu, 25 Feb 2010 00:30:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=HUg5m6EdgBr0cZZRAcez0vlyBFy1cqYgjjwF6CPtNvE=; b=sBNjAtL/s7vacJeNm/CIgluiJTgbfRVpV0hezwERxpPaVvhOW/ljKJUJTXkvn6rfhN 0V08vAxC0BdcoRkkkx+vXnE4u+lyJodNrOcw8kRyeZa95PW9dxjeJ1eAWie7F9aOwsD2 iRNVz/spElNzXArrhHKGRm/IYCTlF9lz7P7FM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=gMYtDybOWtgf0GZ+xcjOXsDRXDVKnZ/HezCN93005RrbXlTcyn51olP3YsflQJJ8dW ZwjNaw8DQc3XdKqxat3V+UQ6jVUIZNXwSSyAcOUaUgO/L1anVhjNXxZBFAjbuj4K2AIc 4VQ2xVgI0kgGnjIus8pC4RvmHpJgvMqb6KW3c= MIME-Version: 1.0 Received: by 10.216.184.205 with SMTP id s55mr419403wem.149.1267085295383; Thu, 25 Feb 2010 00:08:15 -0800 (PST) Date: Thu, 25 Feb 2010 10:08:15 +0200 Message-ID: From: David Naylor To: freebsd-hackers@freebsd.org Content-Type: multipart/mixed; boundary=0016e649864cee75470480684844 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: [Proof of Concept] Stacked unionfs based 'tinderbox' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 08:30:37 -0000 --0016e649864cee75470480684844 Content-Type: text/plain; charset=UTF-8 Hi, As some may have noticed on -current I have been working on using stacked unionfs to implement a 'tinderbox' type build system. I have successfully used the scripts to build x11/xorg (and have compared the results to using the traditional approach using pkg_add). The build system is stable except for one nasty corner case: deadlocks. To setup a compatible test environment requires: - recompile the kernel with `options KSTACK_PAGES=32`, otherwise the kernel will panic with a double fault. WITNESS options results in substantial performance degradation. - patch mtree (see below) [optional] - create the appropriate chroot environment (and reboot) [see below for corner case] A performance bottleneck in mtree was identified. This resulted in mtree (as run by port install) consuming ~20% of the build time. See bin/143732 for a patch and further details. Here are my build times (in the chroot): # time make -C /usr/ports/x11/xorg install clean 2397.42 real 2228.35 user 1151.00 sys # time ./ports-union-builder.sh (old mtree) 8123.25 real 2280.53 user 6319.77 sys # time ./ports-union-builder.sh (new mtree) 6456.11 real 2272.07 user 5778.74 sys # time ./ports-tinder-builder.sh (new mtree) 4270.82 real 2961.88 user 1636.27 sys The "new mtree" is the mtree with the above mentioned patch applied. ports-tinder-builder is an approximate to how the read tinderbox works. The normal tinderbox approach takes ~80% more time to install compared to the quick and dirty approach. The stacked unionfs approach takes ~170% more time (an increase of ~50% over the tinderbox approach). Some performance gains can be had if one uses memory backed storage (vs HDD in this case). This approach should have application where wearing of the underlying storage (such as SDD) is of concern. Further investigation suggests that stacking unionfs scales exponentially, not linearly. This may be an area for further improvements. The corner case: sometimes the directories involved experience a deadlock. Any IO under those directories stall the program. I have found that deleting files in the chroot prior to running the unionfs script reliably causes the deadlock. Since WITNESS had significant performance impact the only data into the deadlocks come from procstat. Below are a variety of procstat -kk calls for deadlocked programs: 68217 100119 pkg-config - mi_switch+0x16f sleepq_wait+0x3b __lockmgr_args+0x641 ffs_lock+0x8c VOP_LOCK1_APV+0x46 unionfs_lock+0x28a VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 6574 100201 ls - mi_switch+0x16f sleepq_wait+0x3b __lockmgr_args+0x654 ffs_lock+0x8c VOP_LOCK1_APV+0x46 _vn_lock+0x44 vget+0x67 cache_lookup+0x4f8 vfs_cache_lookup+0xad VOP_LOOKUP_APV+0x40 lookup+0x44d namei+0x4ec kern_statat_vnhook+0x82 kern_statat+0x15 lstat+0x22 syscall+0x1e7 Xfast_syscall+0xe1 Is anyone able to fix the deadlocking issue? Regards, David. P.S. This has been tested under 8 and 9. P.P.S. The scripts used are attached --0016e649864cee75470480684844-- From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 10:58:24 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E06121065672 for ; Thu, 25 Feb 2010 10:58:24 +0000 (UTC) (envelope-from ray@dlink.ua) Received: from dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id 68ED18FC17 for ; Thu, 25 Feb 2010 10:58:23 +0000 (UTC) Received: from gw ([192.168.10.10] helo=terran) by dlink.ua with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1NkbNs-0004oi-Dp; Thu, 25 Feb 2010 12:55:56 +0200 Date: Thu, 25 Feb 2010 12:58:38 +0200 From: Alexandr Rybalko To: Peter Steele Message-Id: <20100225125838.13cd6c0e.ray@dlink.ua> In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> Organization: D-Link X-Mailer: Sylpheed 2.7.1 (GTK+ 2.16.6; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 10:58:25 -0000 On Wed, 24 Feb 2010 15:56:35 -0600 Peter Steele wrote: >> >> How do I get libc built with full debug symbols? >> > >> >I haven't tried it by myself but think here is the way to go: put the following to /etc/make.conf and recompile needed >> >libraries / ports. WITH_DEBUG=yes >> >DEBUG_FLAGS=-g >> >> That didn't seem to have any effect. I still see -O2 being used instead of -O0. Try to use make DEBUG_FLAGS=-g WITH_DEBUG=yes buildkworld make DEBUG_FLAGS=-g WITH_DEBUG=yes buildkernel >> >> >Mmm... Do other daemons (sshd, lpd, ...) also fail when started through this script? Normal commands (ls, ps) seem not >> >affected. >> >> I tried a few other things and they all seemed to run correctly. We use this same general approach in the full version of this >> script to launch lots of applications. Its role in fact is a process launcher/monitor. I stripped it down to the bare minimum >> in order to isolate the cause of the problem. It seems that only ntpd hangs, but not if I use Python 2.5.1. >> I think problem not in ntpd, since I use ntpdate. And in 50% times, when it run from startup script, it hangs with kernel. No Ctrl+C work, kernel don`t answer for ping, just freeze. Problem somewhere in kernel, maybe in subsystems that set new time, maybe in network(UDP) parts. This problem don`t affect other programs, so I think this in time handling code. Peter, what platform You use? I use MIPS BCM5354. >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Alexandr Rybalko aka Alex RAY From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 13:53:07 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D1F91065672 for ; Thu, 25 Feb 2010 13:53:07 +0000 (UTC) (envelope-from uqs@FreeBSD.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2001:470:9a47::1]) by mx1.freebsd.org (Postfix) with ESMTP id 928B58FC1B for ; Thu, 25 Feb 2010 13:53:06 +0000 (UTC) Received: from acme.spoerlein.net (localhost.spoerlein.net [IPv6:::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id o1PDr4Tf025140 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Feb 2010 14:53:04 +0100 (CET) (envelope-from uqs@FreeBSD.org) Received: (from uqs@localhost) by acme.spoerlein.net (8.14.4/8.14.4/Submit) id o1PDr3fq025137; Thu, 25 Feb 2010 14:53:04 +0100 (CET) (envelope-from uqs@FreeBSD.org) Date: Thu, 25 Feb 2010 14:53:03 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: David Naylor Message-ID: <20100225135303.GM57731@acme.spoerlein.net> Mail-Followup-To: David Naylor , freebsd-hackers@freebsd.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on acme.spoerlein.net Cc: freebsd-hackers@FreeBSD.org Subject: Re: [Proof of Concept] Stacked unionfs based 'tinderbox' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 13:53:07 -0000 On Thu, 25.02.2010 at 10:08:15 +0200, David Naylor wrote: > Hi, > > As some may have noticed on -current I have been working on using > stacked unionfs to implement a 'tinderbox' type build system. I have > successfully used the scripts to build x11/xorg (and have compared the > results to using the traditional approach using pkg_add). The build > system is stable except for one nasty corner case: deadlocks. When I did this a couple of years ago, the major problems were failing chdir(2) calls during ports build, etc. > To setup a compatible test environment requires: > - recompile the kernel with `options KSTACK_PAGES=32`, otherwise the > kernel will panic with a double fault. WITNESS options results in > substantial performance degradation. > - patch mtree (see below) [optional] > - create the appropriate chroot environment (and reboot) [see below > for corner case] > > A performance bottleneck in mtree was identified. This resulted in > mtree (as run by port install) consuming ~20% of the build time. See > bin/143732 for a patch and further details. Good work! > The normal tinderbox approach takes ~80% more time to install compared to the > quick and dirty approach. The stacked unionfs approach takes ~170% more time > (an increase of ~50% over the tinderbox approach). Some performance gains can > be had if one uses memory backed storage (vs HDD in this case). Please explain: what is the quick and dirty approach and which one is faster now? As your scripts did not make it through, perhaps you can upload them to the wiki? What I did back then was using a clean base system as the underlying unionfs store to avoid re-generating the clean base over and over again. Nowadays, a ZFS clone would probably be the way to go. I'm not sure if a recursive approach is feasible here, as you can have only one underlying unionfs mount. But special casing, e.g., perl may still give a massive speedup. So for each port that has perl as dependancy, you would not pull in the clean base + pkg_add perl, but instead grab the clean-base+perl directory as an underlying unionfs. Cheers Uli From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 14:19:08 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5D38106564A for ; Thu, 25 Feb 2010 14:19:08 +0000 (UTC) (envelope-from shuvaev@physik.uni-wuerzburg.de) Received: from mailrelay.rz.uni-wuerzburg.de (mailrelay.rz.uni-wuerzburg.de [132.187.3.28]) by mx1.freebsd.org (Postfix) with ESMTP id 82CDB8FC13 for ; Thu, 25 Feb 2010 14:19:08 +0000 (UTC) Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 4A2095AD64; Thu, 25 Feb 2010 15:19:07 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 4402D5AD41; Thu, 25 Feb 2010 15:19:07 +0100 (CET) X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de Received: from mail.physik.uni-wuerzburg.de (wthp192.physik.uni-wuerzburg.de [132.187.40.192]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 2499A5D7EC; Thu, 25 Feb 2010 15:19:07 +0100 (CET) Received: from wep4035 ([132.187.37.35]) by mail.physik.uni-wuerzburg.de (Lotus Domino Release 8.5.1FP1) with ESMTP id 2010022515190534-61167 ; Thu, 25 Feb 2010 15:19:05 +0100 Received: by wep4035 (sSMTP sendmail emulation); Thu, 25 Feb 2010 15:19:05 +0100 Date: Thu, 25 Feb 2010 15:19:05 +0100 From: Alexey Shuvaev To: Peter Steele Message-ID: <20100225141905.GA31703@wep4035.physik.uni-wuerzburg.de> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> MIME-Version: 1.0 In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> Organization: Universitaet Wuerzburg User-Agent: Mutt/1.5.20 (2009-06-14) X-MIMETrack: Itemize by SMTP Server on domino1/uni-wuerzburg(Release 8.5.1FP1|January 05, 2010) at 02/25/2010 03:19:06 PM, Serialize by Router on domino1/uni-wuerzburg(Release 8.5.1FP1|January 05, 2010) at 02/25/2010 03:19:06 PM, Serialize complete at 02/25/2010 03:19:06 PM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Cc: "freebsd-hackers@freebsd.org" Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 14:19:08 -0000 On Wed, Feb 24, 2010 at 03:56:35PM -0600, Peter Steele wrote: > >> How do I get libc built with full debug symbols? > > > >I haven't tried it by myself but think here is the way to go: put the following to /etc/make.conf and recompile needed libraries / ports. > >WITH_DEBUG=yes > >DEBUG_FLAGS=-g > > That didn't seem to have any effect. I still see -O2 being used > instead of -O0. > The flag you should look at is '-g'. GCC supports debuggind symbols together with -O2 optimizations. Others have posted suggenstions how to build libraries with debugging symbols which go in the same direction. However, with the above variables in make.conf you do not need to remember all the places where you have to put DEBUG_FLAGS=-g in the command line. Just normal buildworld and buildkernel targets will dtrt. That is, you will get the complete base system with debug symbols. The variable WITH_DEBUG=yes is for the software from ports. Just FYI. > >Mmm... Do other daemons (sshd, lpd, ...) also fail when started > through this script? Normal commands (ls, ps) seem not affected. > > I tried a few other things and they all seemed to run correctly. > We use this same general approach in the full version of this script > to launch lots of applications. Its role in fact is a process > launcher/monitor. I stripped it down to the bare minimum in order > to isolate the cause of the problem. It seems that only ntpd hangs, > but not if I use Python 2.5.1. > From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 14:33:46 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56D2D1065674 for ; Thu, 25 Feb 2010 14:33:46 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 13AD88FC24 for ; Thu, 25 Feb 2010 14:33:45 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id D28081FFC4D; Thu, 25 Feb 2010 14:33:44 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id A83FA84500; Thu, 25 Feb 2010 15:33:44 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Alexey Shuvaev References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225141905.GA31703@wep4035.physik.uni-wuerzburg.de> Date: Thu, 25 Feb 2010 15:33:44 +0100 In-Reply-To: <20100225141905.GA31703@wep4035.physik.uni-wuerzburg.de> (Alexey Shuvaev's message of "Thu, 25 Feb 2010 15:19:05 +0100") Message-ID: <86aauxe65z.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-hackers@freebsd.org" , Peter Steele Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 14:33:46 -0000 Alexey Shuvaev writes: > The flag you should look at is '-g'. GCC supports debuggind symbols > together with -O2 optimizations. It is generally not a good idea to use -O2 for debugging versions, since gcc will optimize away many local variables. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 15:12:08 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43F42106564A for ; Thu, 25 Feb 2010 15:12:08 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505c.appriver.com [98.129.35.7]) by mx1.freebsd.org (Postfix) with ESMTP id 070E28FC1A for ; Thu, 25 Feb 2010 15:12:07 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.15 X-Note-Reverse-DNS: ht02.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.15] (HELO ht02.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 29513408; Thu, 25 Feb 2010 08:12:05 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by ht02.exg5.exghost.com ([98.129.23.15]) with mapi; Thu, 25 Feb 2010 08:12:06 -0600 From: Peter Steele To: Alexandr Rybalko Date: Thu, 25 Feb 2010 08:12:05 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acq2CXPTCu4jLrruT7KPSaB3LS71gwAGoqTw Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225125838.13cd6c0e.ray@dlink.ua> In-Reply-To: <20100225125838.13cd6c0e.ray@dlink.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "hackers@freebsd.org" Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 15:12:08 -0000 >I think problem not in ntpd, since I use ntpdate. And in 50% times, when i= t run from startup script, it hangs with kernel. >No Ctrl+C work, kernel don`t answer for ping, just freeze. >Problem somewhere in kernel, maybe in subsystems that set new time, maybe = in network(UDP) parts. >This problem don`t affect other programs, so I think this in time handling= code. I think you may be describing a different problem. For one thing, we don't = use ntpdate, we use the "ntpd -g -q" alternative. Secondly, for us ntpd is = hanging 100% of the time when run via a Python thread class. The exception = is Python 2.5.1; this succeeds 100% of the time. >Peter, what platform You use? I use MIPS BCM5354. We have a variety of 1U and 3U boxes. They all hang the same way. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 15:18:35 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E8391065674 for ; Thu, 25 Feb 2010 15:18:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A46BF8FC3B for ; Thu, 25 Feb 2010 15:18:33 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1PFISXd026898 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Feb 2010 17:18:28 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o1PFIS23070859; Thu, 25 Feb 2010 17:18:28 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o1PFISbQ070839; Thu, 25 Feb 2010 17:18:28 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 25 Feb 2010 17:18:28 +0200 From: Kostik Belousov To: Peter Steele Message-ID: <20100225151828.GA2489@deviant.kiev.zoral.com.ua> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225125838.13cd6c0e.ray@dlink.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4Ckj6UjgE2iN1+kY" Content-Disposition: inline In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Alexandr Rybalko , "hackers@freebsd.org" Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 15:18:35 -0000 --4Ckj6UjgE2iN1+kY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Feb 25, 2010 at 08:12:05AM -0600, Peter Steele wrote: > >I think problem not in ntpd, since I use ntpdate. And in 50% times, when= it run from startup script, it hangs with kernel. > >No Ctrl+C work, kernel don`t answer for ping, just freeze. > >Problem somewhere in kernel, maybe in subsystems that set new time, mayb= e in network(UDP) parts. > >This problem don`t affect other programs, so I think this in time handli= ng code. >=20 > I think you may be describing a different problem. For one thing, we don'= t use ntpdate, we use the "ntpd -g -q" alternative. Secondly, for us ntpd i= s hanging 100% of the time when run via a Python thread class. The exceptio= n is Python 2.5.1; this succeeds 100% of the time. >=20 > >Peter, what platform You use? I use MIPS BCM5354. >=20 > We have a variety of 1U and 3U boxes. They all hang the same way. Very wild guess, check the process signal mask of the child for both methods of spawning. --4Ckj6UjgE2iN1+kY Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkuGlMQACgkQC3+MBN1Mb4iFjwCg3qaheJHGXE6EJcq3Gjpd38Xp vRAAn3zxTwK0yrbjYlbV8HklGZnCGJ7x =fEZL -----END PGP SIGNATURE----- --4Ckj6UjgE2iN1+kY-- From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 15:29:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 870B2106566C for ; Thu, 25 Feb 2010 15:29:31 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: from mail-ew0-f226.google.com (mail-ew0-f226.google.com [209.85.219.226]) by mx1.freebsd.org (Postfix) with ESMTP id 1CC1A8FC12 for ; Thu, 25 Feb 2010 15:29:30 +0000 (UTC) Received: by ewy26 with SMTP id 26so309106ewy.3 for ; Thu, 25 Feb 2010 07:29:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=oCfGaV7eurfHZTLdm+JhCmjvCNm9k7KTlXwfaGTgXns=; b=GAqmCJ+JPUYFjwG+7YNamPFwEMlDAs1qfmLKZcQjBfhe7azxYs8NztQuGmfuSGH2tR 3HJPNmRAMlJYkn97ZJbMSFUN4v6a97FPHKmlesMUVSkqrNybN53ycL4goi4g1zuMNL+B JJpSczRVgNcdrsdgih7t+xQufMsC1fUv8KEVo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=mn8zC4lLOMKzJC03eJd64/pEFGDPbgZqw7FzJWCj5VAPFFDQsf445m5pnrRaf6L8ZR qsnXa2ReeUF2Az+FNh9f6iIIHDgpL8zzbw73g+TjdGsxv8kpWY16xVVKqma+jXkN/yeM Vxq0+R7wFjPIp+DgL2UreZ4jsmvPMUm4FffhQ= MIME-Version: 1.0 Received: by 10.216.182.10 with SMTP id n10mr662149wem.142.1267111760766; Thu, 25 Feb 2010 07:29:20 -0800 (PST) In-Reply-To: <20100225135303.GM57731@acme.spoerlein.net> References: <20100225135303.GM57731@acme.spoerlein.net> Date: Thu, 25 Feb 2010 17:29:19 +0200 Message-ID: From: David Naylor To: David Naylor , freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: [Proof of Concept] Stacked unionfs based 'tinderbox' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 15:29:31 -0000 On 25 February 2010 15:53, Ulrich Sp=C3=B6rlein wrote: > On Thu, 25.02.2010 at 10:08:15 +0200, David Naylor wrote: >> Hi, >> >> As some may have noticed on -current I have been working on using >> stacked unionfs to implement a 'tinderbox' type build system. =C2=A0I ha= ve >> successfully used the scripts to build x11/xorg (and have compared the >> results to using the traditional approach using pkg_add). =C2=A0The buil= d >> system is stable except for one nasty corner case: deadlocks. > > When I did this a couple of years ago, the major problems were failing > chdir(2) calls during ports build, etc. > >> To setup a compatible test environment requires: >> =C2=A0- recompile the kernel with `options KSTACK_PAGES=3D32`, otherwise= the >> =C2=A0kernel will panic with a double fault. =C2=A0WITNESS options resul= ts in >> =C2=A0substantial performance degradation. >> =C2=A0- patch mtree (see below) [optional] >> =C2=A0- create the appropriate chroot environment (and reboot) [see belo= w >> =C2=A0for corner case] >> >> A performance bottleneck in mtree was identified. =C2=A0This resulted in >> mtree (as run by port install) consuming ~20% of the build time. =C2=A0S= ee >> bin/143732 for a patch and further details. > > Good work! > >> The normal tinderbox approach takes ~80% more time to install compared t= o the >> quick and dirty approach. =C2=A0The stacked unionfs approach takes ~170%= more time >> (an increase of ~50% over the tinderbox approach). =C2=A0Some performanc= e gains can >> be had if one uses memory backed storage (vs HDD in this case). > > Please explain: what is the quick and dirty approach and which one is > faster now? The quick and dirty is `make -C /usr/ports/x11/xorg install clean`. The stacked unionfs is still the slowest (even with a 20% improvement from patching mtree). > As your scripts did not make it through, perhaps you can upload them to > the wiki? What I did back then was using a clean base system as the > underlying unionfs store to avoid re-generating the clean base over and > over again. Nowadays, a ZFS clone would probably be the way to go. > > I'm not sure if a recursive approach is feasible here, as you can have > only one underlying unionfs mount. But special casing, e.g., perl may > still give a massive speedup. So for each port that has perl as > dependancy, you would not pull in the clean base + pkg_add perl, but > instead grab the clean-base+perl directory as an underlying unionfs. > > Cheers > Uli > From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 15:57:51 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 868DB1065677 for ; Thu, 25 Feb 2010 15:57:51 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505b.appriver.com [98.129.35.5]) by mx1.freebsd.org (Postfix) with ESMTP id 46D0F8FC16 for ; Thu, 25 Feb 2010 15:57:50 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Policy: GLOBAL - maxiscale.com X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.45 X-Note-Reverse-DNS: ht03.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.45] (HELO HT03.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 25637993; Thu, 25 Feb 2010 09:57:50 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by HT03.exg5.exghost.com ([10.242.228.75]) with mapi; Thu, 25 Feb 2010 09:57:49 -0600 From: Peter Steele To: Kostik Belousov Date: Thu, 25 Feb 2010 09:57:48 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acq2LdGqyjdn0SqtRcqL6+F8j/wPewABMFPg Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95703@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225125838.13cd6c0e.ray@dlink.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> <20100225151828.GA2489@deviant.kiev.zoral.com.ua> In-Reply-To: <20100225151828.GA2489@deviant.kiev.zoral.com.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Alexandr Rybalko , "hackers@freebsd.org" Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 15:57:51 -0000 >Very wild guess, check the process signal mask of the child for both metho= ds of spawning. I'm running ntpd through Python. How do I check the process signal mask? I = did some quick searches and it seems Python does not support sigprocmask().= =20 In my searches I came across this link: http://bugs.python.org/msg61870 I think you might be right that this is related to the signal mask. In my s= cenario the select call is hanging indefinitely, just like discussed in thi= s article. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 16:45:23 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23239106564A for ; Thu, 25 Feb 2010 16:45:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 71A538FC13 for ; Thu, 25 Feb 2010 16:45:21 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1PGjFvB033940 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Feb 2010 18:45:15 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o1PGjEQq095606; Thu, 25 Feb 2010 18:45:14 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o1PGjE9a095605; Thu, 25 Feb 2010 18:45:14 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 25 Feb 2010 18:45:14 +0200 From: Kostik Belousov To: Peter Steele Message-ID: <20100225164514.GB2489@deviant.kiev.zoral.com.ua> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225125838.13cd6c0e.ray@dlink.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> <20100225151828.GA2489@deviant.kiev.zoral.com.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95703@MBX03.exg5.exghost.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NDin8bjvE/0mNLFQ" Content-Disposition: inline In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95703@MBX03.exg5.exghost.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Alexandr Rybalko , "hackers@freebsd.org" Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 16:45:23 -0000 --NDin8bjvE/0mNLFQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Feb 25, 2010 at 09:57:48AM -0600, Peter Steele wrote: > >Very wild guess, check the process signal mask of the child for both met= hods of spawning. >=20 > I'm running ntpd through Python. How do I check the process signal mask? = I did some quick searches and it seems Python does not support sigprocmask(= ).=20 >=20 > In my searches I came across this link: >=20 > http://bugs.python.org/msg61870 >=20 > I think you might be right that this is related to the signal mask. In my= scenario the select call is hanging indefinitely, just like discussed in t= his article. >=20 Below is the quickly made patch to add ability to show signal disposition to the procstat(1). I am not sure about duplicating information about catch/ignore state of the signal for all threads (this information is process-global), but I think this is more usable for scripts. diff --git a/usr.bin/procstat/Makefile b/usr.bin/procstat/Makefile index 1c187b0..251fc06 100644 --- a/usr.bin/procstat/Makefile +++ b/usr.bin/procstat/Makefile @@ -10,6 +10,7 @@ SRCS=3D procstat.c \ procstat_files.c \ procstat_kstack.c \ procstat_threads.c \ + procstat_threads_sigs.c \ procstat_vm.c =20 LDADD+=3D -lutil diff --git a/usr.bin/procstat/procstat.c b/usr.bin/procstat/procstat.c index bc02682..cbd4eca 100644 --- a/usr.bin/procstat/procstat.c +++ b/usr.bin/procstat/procstat.c @@ -38,7 +38,7 @@ =20 #include "procstat.h" =20 -static int aflag, bflag, cflag, fflag, kflag, sflag, tflag, vflag; +static int aflag, bflag, cflag, fflag, iflag, kflag, sflag, tflag, vflag; int hflag; =20 static void @@ -46,7 +46,7 @@ usage(void) { =20 fprintf(stderr, "usage: procstat [-h] [-w interval] [-b | -c | -f | " - "-k | -s | -t | -v]\n"); + "-i | -k | -s | -t | -v]\n"); fprintf(stderr, " [-a | pid ...]\n"); exit(EX_USAGE); } @@ -61,6 +61,8 @@ procstat(pid_t pid, struct kinfo_proc *kipp) procstat_args(pid, kipp); else if (fflag) procstat_files(pid, kipp); + else if (iflag) + procstat_threads_sigs(pid, kipp); else if (kflag) procstat_kstack(pid, kipp, kflag); else if (sflag) @@ -109,7 +111,7 @@ main(int argc, char *argv[]) char *dummy; =20 interval =3D 0; - while ((ch =3D getopt(argc, argv, "abcfkhstvw:")) !=3D -1) { + while ((ch =3D getopt(argc, argv, "abcfikhstvw:")) !=3D -1) { switch (ch) { case 'a': aflag++; @@ -127,6 +129,10 @@ main(int argc, char *argv[]) fflag++; break; =20 + case 'i': + iflag++; + break; + case 'k': kflag++; break; diff --git a/usr.bin/procstat/procstat.h b/usr.bin/procstat/procstat.h index 8bacab7..10f8fce 100644 --- a/usr.bin/procstat/procstat.h +++ b/usr.bin/procstat/procstat.h @@ -41,6 +41,7 @@ void procstat_cred(pid_t pid, struct kinfo_proc *kipp); void procstat_files(pid_t pid, struct kinfo_proc *kipp); void procstat_kstack(pid_t pid, struct kinfo_proc *kipp, int kflag); void procstat_threads(pid_t pid, struct kinfo_proc *kipp); +void procstat_threads_sigs(pid_t pid, struct kinfo_proc *kipp); void procstat_vm(pid_t pid, struct kinfo_proc *kipp); =20 #endif /* !PROCSTAT_H */ diff --git a/usr.bin/procstat/procstat_threads_sigs.c b/usr.bin/procstat/pr= ocstat_threads_sigs.c new file mode 100644 index 0000000..814f0c4 --- /dev/null +++ b/usr.bin/procstat/procstat_threads_sigs.c @@ -0,0 +1,111 @@ +/*- + * Copyright (c) 2007 Robert N. M. Watson + * Copyright (c) 2010 Konstantin Belousov + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#include "procstat.h" + +static inline void +procstat_print_sig(const sigset_t *set, int sig) +{ + + printf("%4s ", sigismember(set, sig) ? "t" : "f"); +} + +void +procstat_threads_sigs(pid_t pid, struct kinfo_proc *kipp) +{ + struct kinfo_proc *kip; + int error, name[4], j; + char rtname[8]; + unsigned int i; + size_t len; + + if (!hflag) + printf("%5s %6s %7s %4s %4s %4s %4s\n", "PID", + "TID", "SIG", "PEND", "BLKC", "IGNR", "CTCH"); + + /* + * We need to re-query for thread information, so don't use *kipp. + */ + name[0] =3D CTL_KERN; + name[1] =3D KERN_PROC; + name[2] =3D KERN_PROC_PID | KERN_PROC_INC_THREAD; + name[3] =3D pid; + + len =3D 0; + error =3D sysctl(name, 4, NULL, &len, NULL, 0); + if (error < 0 && errno !=3D ESRCH) { + warn("sysctl: kern.proc.pid: %d", pid); + return; + } + if (error < 0) + return; + + kip =3D malloc(len); + if (kip =3D=3D NULL) + err(-1, "malloc"); + + if (sysctl(name, 4, kip, &len, NULL, 0) < 0) { + warn("sysctl: kern.proc.pid: %d", pid); + free(kip); + return; + } + + kinfo_proc_sort(kip, len / sizeof(*kipp)); + for (i =3D 0; i < len / sizeof(*kipp); i++) { + kipp =3D &kip[i]; + for (j =3D 1; j < _SIG_MAXSIG; j++) { + printf("%5d ", pid); + printf("%6d ", kipp->ki_tid); + if (j < sys_nsig) + printf("%7s ", sys_signame[j]); + else if (j >=3D SIGRTMIN && j <=3D SIGRTMAX) { + snprintf(rtname, sizeof(rtname), "rt%d", j); + printf("%7s ", rtname); + } else + printf("%7d ", j); + procstat_print_sig(&kipp->ki_siglist, j); + procstat_print_sig(&kipp->ki_sigmask, j); + procstat_print_sig(&kipp->ki_sigignore, j); + procstat_print_sig(&kipp->ki_sigcatch, j); + printf("\n"); + } + } + free(kip); +} --NDin8bjvE/0mNLFQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkuGqRoACgkQC3+MBN1Mb4hZ0gCgvRRR8hhBzZPcLRtkjkUJWAE9 u3gAoMtmruhpGnvvMOkInWB+oY7570BH =1nhM -----END PGP SIGNATURE----- --NDin8bjvE/0mNLFQ-- From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 16:57:25 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A844D1065670 for ; Thu, 25 Feb 2010 16:57:25 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505a.appriver.com [98.129.35.4]) by mx1.freebsd.org (Postfix) with ESMTP id 68E158FC13 for ; Thu, 25 Feb 2010 16:57:25 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.45 X-Note-Reverse-DNS: ht03.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.45] (HELO HT03.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 29555459; Thu, 25 Feb 2010 10:57:25 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by HT03.exg5.exghost.com ([10.242.228.75]) with mapi; Thu, 25 Feb 2010 10:57:23 -0600 From: Peter Steele To: Kostik Belousov Date: Thu, 25 Feb 2010 10:57:22 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acq2OfW9mRE3bd/fQzSGLzlKnSS2OAAAKTSQ Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95765@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225125838.13cd6c0e.ray@dlink.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> <20100225151828.GA2489@deviant.kiev.zoral.com.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95703@MBX03.exg5.exghost.com> <20100225164514.GB2489@deviant.kiev.zoral.com.ua> In-Reply-To: <20100225164514.GB2489@deviant.kiev.zoral.com.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "hackers@freebsd.org" Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 16:57:25 -0000 >Below is the quickly made patch to add ability to show signal disposition = to the procstat(1). I am not sure about duplicating information about catch= /ignore state >of the signal for allthreads (this information is process-global), but I t= hink this is more usable for scripts. Thanks for the patch. To be honest though, this issue has already cost us w= ay too much time. We've got a workaround by either going with Python 2.5.1 = or by running ntpd via the "at" command: os.system("echo ntpd -g -q|at now") If we running ntpd in this fashion it doesn't hang, more proof that the pro= blem is related to Python's manipulation of the process signal mask. Runnin= g it through the "at" subsystem removes it from the Python process hierarch= y. We'll likely go with this solution instead of downgrading Python and the= related libraries. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 16:58:48 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DED97106564A for ; Thu, 25 Feb 2010 16:58:48 +0000 (UTC) (envelope-from nate@thatsmathematics.com) Received: from euclid.ucsd.edu (euclid.ucsd.edu [132.239.145.52]) by mx1.freebsd.org (Postfix) with ESMTP id B63D28FC13 for ; Thu, 25 Feb 2010 16:58:48 +0000 (UTC) Received: from zeno.ucsd.edu (zeno.ucsd.edu [132.239.145.22]) by euclid.ucsd.edu (8.11.7p3+Sun/8.11.7) with ESMTP id o1PGwmY12841; Thu, 25 Feb 2010 08:58:48 -0800 (PST) Received: from localhost (neldredg@localhost) by zeno.ucsd.edu (8.11.7p3+Sun/8.11.7) with ESMTP id o1PGwlh25625; Thu, 25 Feb 2010 08:58:48 -0800 (PST) X-Authentication-Warning: zeno.ucsd.edu: neldredg owned process doing -bs Date: Thu, 25 Feb 2010 08:58:47 -0800 (PST) From: Nate Eldredge X-X-Sender: neldredg@zeno.ucsd.edu To: Peter Steele In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95703@MBX03.exg5.exghost.com> Message-ID: References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225125838.13cd6c0e.ray@dlink.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> <20100225151828.GA2489@deviant.kiev.zoral.com.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95703@MBX03.exg5.exghost.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Kostik Belousov , Alexandr Rybalko , "hackers@freebsd.org" Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 16:58:48 -0000 On Thu, 25 Feb 2010, Peter Steele wrote: >> Very wild guess, check the process signal mask of the child for both >> methods of spawning. > > I'm running ntpd through Python. How do I check the process signal mask? > I did some quick searches and it seems Python does not support > sigprocmask(). That wouldn't help, because you'd get the signal mask of python itself, not of the ntpd child process. truss and/or ktrace can help here, just look backward in the trace for the last call to sigprocmask. (Another possible culprit can be a signal that's handled and then a longjmp (not siglongjmp) out of the handler. You'd see this in the trace as a signal delivery not followed by sigprocmask or sigreturn.) -- Nate Eldredge nate@thatsmathematics.com From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 17:00:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AAE4C106566C for ; Thu, 25 Feb 2010 17:00:54 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 3B4A88FC12 for ; Thu, 25 Feb 2010 17:00:53 +0000 (UTC) Received: by fxm23 with SMTP id 23so351762fxm.3 for ; Thu, 25 Feb 2010 09:00:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:subject :message-id:in-reply-to:references:x-mailer:mime-version :content-type:content-transfer-encoding; bh=ymjFR90npO9pWkCKc9TsrGcw4lNZaU2NO5PhW7/Hbhw=; b=doa5ceQUOQVtqIJuQzQAYZcZ8xItUq9KKQZyLMHKdaXqwiwkFK7d8oj6u54KsL5D6H AR0D7Lqr8bJ6bGMSBloGoK4zkX6UjdgBggBRFzULndApI6ZykjAmZ/JG8plDWvEH11U8 ChJvnK6eYoDmI4zafoNbaXjrIT+Ai/0FtSXNM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type:content-transfer-encoding; b=ORCkQgseznrIC2OL1nrTvwbnrq45zcdojgpyWpMYz0rgmO7laBcZ8KQZOaUC56uqaE skWzGHG3JBRtmRXCv8HNQLhSWHWA8FsxuNf8Wl07YlYM6r3tiBD3vRtu7W++hpfChTDt bQbypgOcy7Z0Mu5l4rvxCRMAQYlKNzxCJ8j5Y= Received: by 10.223.57.133 with SMTP id c5mr1477943fah.11.1267117250155; Thu, 25 Feb 2010 09:00:50 -0800 (PST) Received: from gumby.homeunix.com (bb-87-81-140-128.ukonline.co.uk [87.81.140.128]) by mx.google.com with ESMTPS id y15sm4486469fkd.8.2010.02.25.09.00.47 (version=SSLv3 cipher=RC4-MD5); Thu, 25 Feb 2010 09:00:48 -0800 (PST) Date: Thu, 25 Feb 2010 17:00:46 +0000 From: RW To: freebsd-hackers@freebsd.org Message-ID: <20100225170046.541e4b8c@gumby.homeunix.com> In-Reply-To: <86sk8qpfx8.fsf@ds4.des.no> References: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> <86eikar7gv.fsf@ds4.des.no> <4B858007.1000008@gmail.com> <86sk8qpfx8.fsf@ds4.des.no> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: 2 bytes allocated problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 17:00:54 -0000 On Wed, 24 Feb 2010 20:55:15 +0100 Dag-Erling Sm=F8rgrav wrote: > Andrey Zonov writes: > > Dag-Erling Sm=F8rgrav writes: > > > Pointers have no boundareis in C. > > And how free() finds that the need to release? >=20 > That is a very simple question with a very complicated answer. A simple answer is that information about the amount of memory allocated is stored at the time it's allocated. And free() can find that data from the pointer. I don't know how it works in FreeBSD, but a traditional method is to put the metadata just before the block of memory returned by malloc - so free() can find it with a simple fixed offset. Note that the allocation algorithm is free to allocate more memory than it needs to, it doesn't need to record what malloc actually asked for, and it doesn't need to know about data structures that are put in that memory.=20 From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 19:22:41 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA37A1065670 for ; Thu, 25 Feb 2010 19:22:41 +0000 (UTC) (envelope-from drbaud@yahoo.com) Received: from n6b.bullet.mail.ac4.yahoo.com (n6b.bullet.mail.ac4.yahoo.com [76.13.13.76]) by mx1.freebsd.org (Postfix) with SMTP id 90FA48FC15 for ; Thu, 25 Feb 2010 19:22:41 +0000 (UTC) Received: from [74.6.228.95] by n6.bullet.mail.ac4.yahoo.com with NNFMP; 25 Feb 2010 19:09:38 -0000 Received: from [76.13.10.174] by t2.bullet.mail.ac4.yahoo.com with NNFMP; 25 Feb 2010 19:09:38 -0000 Received: from [127.0.0.1] by omp115.mail.ac4.yahoo.com with NNFMP; 25 Feb 2010 19:09:38 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 702677.38576.bm@omp115.mail.ac4.yahoo.com Received: (qmail 15558 invoked by uid 60001); 25 Feb 2010 19:09:37 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1267124977; bh=RTfvx0zFYHYfE/q6S4w06z4TpCrhOUkS4kMHctl3DW4=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=YvGKOtyhZG/mWrovTwm63hT/f2C8va1NAtWD9/pHmn6st+rfB4PLAgokH38ZqBJsP/RnetxflFxIdmjgv+J4MZeqjGPd5ogLTRFatma3KJLOroaeZ2FIaV6Kingxj8HiDVHc2kBBmfS2068R4oxOu0sYDsu6s++y3KrcomBtSv8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=oDVJnWduyxVq+wBZBTZoB2+g+OMD7jFj9H9c6qCY+R3OlacFkEBs3giLPuCFllNqOH36Wjv2Eo1M8sy9vu3D3E5vbs2XDfVXTxbPkJioOlduwEDjRI9uS6hNa7NF/anGFAIrKWP6NH/6voAAauOID7Jl9LlbG9pIqjIA7KLBpQw=; Message-ID: <87341.15532.qm@web65601.mail.ac4.yahoo.com> X-YMail-OSG: jEb64Z8VM1kmbghTlSUBEJiZdZXFweHb6Au8YijXsVQPP13_hCHQia2ySgFQ.ITBSu1WvV0Tm2s99f7mPJkJ3z_pSQovEbpsnWOwnTLp3A8p.bShwYKRGllSNqFJHOHOeAdidUllGmSz8et8GHKYjJeX.pKvEAC_.F7NR.nzgOHmHOr3bNZcJUb25xY4JjPGctdbZc5sl0Dqg08z5Gm2PXJkMqyniR7bORJPZfrdufra4b7xWt5fVWp63EQ.4.lQUO_RTHuznUhu8NAvTLlveiq8x4AvVumuJ9tnbEQwaR5kmWDRqxcmJR3c Received: from [64.238.244.146] by web65601.mail.ac4.yahoo.com via HTTP; Thu, 25 Feb 2010 11:09:36 PST X-Mailer: YahooMailClassic/9.2.12 YahooMailWebService/0.8.100.260964 Date: Thu, 25 Feb 2010 11:09:36 -0800 (PST) From: "Dr. Baud" To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailman-Approved-At: Thu, 25 Feb 2010 19:40:14 +0000 Subject: debug libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 19:22:42 -0000 Apologies if this is the wrong list.... Are there prepackaged debug versions of the system libraries available (like 'yum install *-debuginfo' in Fedora and 'apt-get install *-dbg' in Ubuntu)? Dr From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 20:30:52 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E9D8106564A; Thu, 25 Feb 2010 20:30:52 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id 0AE338FC1E; Thu, 25 Feb 2010 20:30:51 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id E91551E00173; Thu, 25 Feb 2010 21:30:50 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1PKSoIG079567; Thu, 25 Feb 2010 21:28:50 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1PKSovb079566; Thu, 25 Feb 2010 21:28:50 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Thu, 25 Feb 2010 21:28:50 +0100 To: Juergen Lock Message-ID: <20100225202850.GA79505@triton8.kn-bremen.de> References: <20100223215010.GA67619@triton8.kn-bremen.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100223215010.GA67619@triton8.kn-bremen.de> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Thu, 25 Feb 2010 20:53:32 +0000 Cc: freebsd-hackers@FreeBSD.org, freebsd-emulation@FreeBSD.org, Tim Kientzle Subject: 32 bit Linux lseek missing overflow check (was: Re: Linuxolator patches: stat and lseek SEEK_END for disk devices) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 20:30:52 -0000 On Tue, Feb 23, 2010 at 10:50:10PM +0100, Juergen Lock wrote: > Hi! > > Before this gets buried on -hackers in another thead... :) > > I now have disks appear as block devices for Linux processes (there > already was commented out code for that in linux_stats.c, I hope my > version is now `correct enough' to be usable [1]), and I made a simple > patch to make lseek SEEK_END (L_XTND in the source) dtrt on disk > devices too by simply invoking the DIOCGMEDIASIZE ioctl there; [2] > both of these things are what (some) Linux processes expect. > > Patches are here: (made on stable/8, if they don't apply on head > I'll have to make extra versions for that...) > http://people.freebsd.org/~nox/linuxdisk-blk.patch [1] > http://people.freebsd.org/~nox/lseek-seek_end.patch [2] > > And yes, with these patches the Linux bsdtar mentioned on -hackers > in the `"tar tfv /dev/cd0" speedup patch' thread now also runs fast > on FreeBSD. :) I now added an vn_isdisk() check to the second patch after comments from julian, and I made a new patch that adds an overflow check to the 32 bit linux lseek: (also at http://people.freebsd.org/~nox/linux-lseek-overflow.patch ) Index: src/sys/compat/linux/linux_file.c =================================================================== RCS file: /home/scvs/src/sys/compat/linux/linux_file.c,v retrieving revision 1.119.2.1 diff -u -p -u -p -r1.119.2.1 linux_file.c --- src/sys/compat/linux/linux_file.c 3 Aug 2009 08:13:06 -0000 1.119.2.1 +++ src/sys/compat/linux/linux_file.c 25 Feb 2010 20:08:47 -0000 @@ -226,6 +226,7 @@ linux_lseek(struct thread *td, struct li int whence; } */ tmp_args; int error; + l_off_t l_off; #ifdef DEBUG if (ldebug(lseek)) @@ -236,6 +237,10 @@ linux_lseek(struct thread *td, struct li tmp_args.offset = (off_t)args->off; tmp_args.whence = args->whence; error = lseek(td, &tmp_args); + /* Check for overflow like Linux does. */ + l_off = *(off_t *)td->td_retval; + if (((off_t)l_off) != *(off_t *)td->td_retval) + error = EOVERFLOW; return error; } From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 22:07:20 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2F24106564A for ; Thu, 25 Feb 2010 22:07:20 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 300AD8FC1F for ; Thu, 25 Feb 2010 22:07:19 +0000 (UTC) Received: (qmail invoked by alias); 25 Feb 2010 22:07:17 -0000 Received: from f055003018.adsl.alicedsl.de (EHLO mandree.no-ip.org) [78.55.3.18] by mail.gmx.net (mp059) with SMTP; 25 Feb 2010 23:07:17 +0100 X-Authenticated: #428038 X-Provags-ID: V01U2FsdGVkX1/NiZZzOIHNFZMzIqxDhFabee8s94KfxKemmNa929 m6umeUlORMgEpS Received: from merlin.emma.line.org (localhost [127.0.0.1]) by merlin.emma.line.org (Postfix) with ESMTP id 1C1FF9435F; Thu, 25 Feb 2010 23:07:16 +0100 (CET) Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , "Andrey Zonov" References: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> <86eikar7gv.fsf@ds4.des.no> <4B858007.1000008@gmail.com> <86sk8qpfx8.fsf@ds4.des.no> Date: Thu, 25 Feb 2010 23:07:15 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Matthias Andree" Organization: Message-ID: In-Reply-To: <86sk8qpfx8.fsf@ds4.des.no> User-Agent: Opera Mail/10.10 (Linux) X-Y-GMX-Trusted: 0 X-FuHaFi: 0.62 Cc: freebsd-hackers@freebsd.org Subject: Re: 2 bytes allocated problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 22:07:21 -0000 Am 24.02.2010, 20:55 Uhr, schrieb Dag-Erling Smørgrav : > Why is there a 0 after the 'i'? Because when you write "abcdefghi", the > compiler actually stores "abcdefghi\0". That's the definition of > "string" in C: a sequence of characters immediately followed by a 0. If > you don't want the 0 there, you have to do something like this: > > char a[9] = { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i' }; char a[9] = "abcdefghi"; suffices. The compiler knows there isn't room for the terminal '\0' and omits it. char a[] = "abcdefghi"; would append the implicit \0 and reserve 10 bytes for a. > but then you don't have a string, just an array of 9 chars. Not that the compiler itself could/would tell the difference after initialization, or that it would even care. It's the library functions that expect strings that care about the \0. Beyond that, I recommend a C book to Andrey. -- Matthias Andree From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 22:26:27 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B7367106564A for ; Thu, 25 Feb 2010 22:26:26 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505b.appriver.com [98.129.35.5]) by mx1.freebsd.org (Postfix) with ESMTP id 789F98FC16 for ; Thu, 25 Feb 2010 22:26:25 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.14 X-Note-Reverse-DNS: ht01.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.14] (HELO ht01.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 25725728 for freebsd-hackers@freebsd.org; Thu, 25 Feb 2010 16:26:23 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by ht01.exg5.exghost.com ([98.129.23.14]) with mapi; Thu, 25 Feb 2010 16:26:23 -0600 From: Peter Steele To: "freebsd-hackers@freebsd.org" Date: Thu, 25 Feb 2010 16:26:22 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acq2OfW9mRE3bd/fQzSGLzlKnSS2OAAAKTSQAAuB6ZA= Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95991@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225125838.13cd6c0e.ray@dlink.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> <20100225151828.GA2489@deviant.kiev.zoral.com.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95703@MBX03.exg5.exghost.com> <20100225164514.GB2489@deviant.kiev.zoral.com.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95765@MBX03.exg5.exghost.com> In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95765@MBX03.exg5.exghost.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 22:26:27 -0000 > We'll likely go with this solution instead of downgrading Python and the = related libraries. In fact I came up with another solution. I realized that since the problem = was related to the process signal mask, instead of called ntpd directly, wr= ap it up in a C app that resets the signal mask to something that works. I = have the following code: sigset_t set, oset; sigemptyset(&set); pthread_sigmask(SIG_SETMASK, &set, &oset); system("/usr/sbin/ntpd -g -q"); pthread_sigmask(SIG_SETMASK, &oset, NULL); I wrapped this up into a standalone app and call this from Python instead o= f calling ntpd directly. This solved the problem--no more hang. Thanks very= much to Kostik Belousov for his "wild guess" that this was related to the = process signal mask. His guess was dead on. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 22:46:04 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D02CC106564A for ; Thu, 25 Feb 2010 22:46:04 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 91AD98FC18 for ; Thu, 25 Feb 2010 22:46:04 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 748A01FFC22; Thu, 25 Feb 2010 22:46:03 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 4B9F2844CC; Thu, 25 Feb 2010 23:46:03 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: "Matthias Andree" References: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> <86eikar7gv.fsf@ds4.des.no> <4B858007.1000008@gmail.com> <86sk8qpfx8.fsf@ds4.des.no> Date: Thu, 25 Feb 2010 23:46:03 +0100 In-Reply-To: (Matthias Andree's message of "Thu, 25 Feb 2010 23:07:15 +0100") Message-ID: <86vddlaq8k.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Andrey Zonov , freebsd-hackers@freebsd.org Subject: Re: 2 bytes allocated problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 22:46:04 -0000 "Matthias Andree" writes: > Dag-Erling Sm=C3=B8rgrav writes: > > char a[9] =3D { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i' }; > char a[9] =3D "abcdefghi"; > > suffices. The compiler knows there isn't room for the terminal '\0' > and omits it. Some compilers (gcc at least) warn about it. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 22:53:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7146B106566C for ; Thu, 25 Feb 2010 22:53:54 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by mx1.freebsd.org (Postfix) with ESMTP id 01B4B8FC16 for ; Thu, 25 Feb 2010 22:53:53 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-066-018-168.pools.arcor-ip.net [88.66.18.168]) by mrelayeu.kundenserver.de (node=mrbap1) with ESMTP (Nemesis) id 0Ld3uy-1NKDDM3JA0-00i7UU; Thu, 25 Feb 2010 23:53:52 +0100 Received: (qmail 22854 invoked from network); 25 Feb 2010 22:53:52 -0000 Received: from f8x64.laiers.local (192.168.4.188) by ns1.laiers.local with SMTP; 25 Feb 2010 22:53:52 -0000 From: Max Laier Organization: FreeBSD To: freebsd-hackers@freebsd.org Date: Thu, 25 Feb 2010 23:53:51 +0100 User-Agent: KMail/1.12.4 (FreeBSD/8.0-RELEASE-p2; KDE/4.3.5; amd64; ; ) References: <983a1cf21002240544s59006035ifbf0ef7eb045e44f@mail.gmail.com> <86vddlaq8k.fsf@ds4.des.no> In-Reply-To: <86vddlaq8k.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201002252353.51573.max@love2party.net> X-Provags-ID: V01U2FsdGVkX18D+DedhlxunIiZpm/QomWjzpwD61xqmu4CbfH +NxyhwtABaEM2iJg8CXOeVs68h/un/8S565NYOo0B0rwTpiAgb RFTJs+MvOw6vNF7J41wmg== Cc: Andrey Zonov , Dag-Erling =?utf-8?q?Sm=C3=B8rgrav?= , Matthias Andree Subject: Re: 2 bytes allocated problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 22:53:54 -0000 On Thursday 25 February 2010 23:46:03 Dag-Erling Sm=C3=B8rgrav wrote: > "Matthias Andree" writes: > > Dag-Erling Sm=C3=B8rgrav writes: > > > char a[9] =3D { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i' }; > > > > char a[9] =3D "abcdefghi"; > > > > suffices. The compiler knows there isn't room for the terminal '\0' > > and omits it. >=20 > Some compilers (gcc at least) warn about it. It shouldn't. C99 explicitly has this as an example (6.7.8.32). It's also= of=20 note that char *p =3D "abc" is different from char a[] =3D "abc" in that the elements of a are modifiable, while modifying p[x] yields=20 undefined behavior. But this is increasingly getting out of scope for hackers@ ... so I'll shut= up=20 now. =2D- Max From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 23:53:04 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85D84106566B for ; Thu, 25 Feb 2010 23:53:04 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by mx1.freebsd.org (Postfix) with ESMTP id 4BE108FC0C for ; Thu, 25 Feb 2010 23:53:04 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.4/8.14.4) with ESMTP id o1PNr3PM028195 for ; Thu, 25 Feb 2010 15:53:03 -0800 (PST) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.4/8.14.4/Submit) id o1PNr3Xb028194 for freebsd-hackers@freebsd.org; Thu, 25 Feb 2010 15:53:03 -0800 (PST) (envelope-from sgk) Date: Thu, 25 Feb 2010 15:53:03 -0800 From: Steve Kargl To: freebsd-hackers@freebsd.org Message-ID: <20100225235303.GA28105@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Mailman-Approved-At: Fri, 26 Feb 2010 00:13:01 +0000 Subject: Nasty bug in jn(3) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 23:53:04 -0000 There's a nasty little bug lurking in jn(3). #include #include int main(void) { double z; int i, n; z = 2.4048255576957729; for (n = 2; n < 10; n++) printf("%d %e\n", n, jn(n,z)); return (0); } troutmask:kargl[446] cc -o z testjn.c -lm troutmask:kargl[447] ./z 2 4.317548e-01 3 -inf 4 4.069027e-02 5 -inf 6 3.247664e-03 7 -inf 8 7.495602e-05 9 -inf I can assure you that -inf is not a valid value for an integer order Bessel function at z = 2.40482555... A quick inspection of e_jn.c suggest a similar problem maybe found at other zeros of j0(x). -- Steve From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 02:50:52 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 988B6106566C for ; Fri, 26 Feb 2010 02:50:52 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7380D8FC1C for ; Fri, 26 Feb 2010 02:50:52 +0000 (UTC) Received: by pvg3 with SMTP id 3so1604610pvg.13 for ; Thu, 25 Feb 2010 18:50:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=iCXqZlfyB5ueDwxCCT6mSwdqUA2Ji3u2W05de1p0bvY=; b=JQH9Dvx/9+ykJgmeHvIYf7BYfk6vLX0A8Wy/oZryDfYSfkktFqbQdGOBYEnCMVNB/J p13p2XE+YaF/7mg+92dAoA/a+2VsQ7vu7kQvuGB0ZGrCsPgfs8RBgQS7SXov+aPppqjc IysInhJk81myatQzv5rqa+PK44e6nJiRWUsv8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=G6wbCPLxla9bfvmtzVIwsj/FKseHqDL6Mi7ypmC0CLKxxOMUe8im05Ju0bwz15yrZM mqfjaEEnnvx7vM1Pb4e3OmSIAPFyuilf0Vn0TIwW8vQN2uy+I8i/uNQq8MgZhp6ugbqD VwPWC+t9esECQSQqyk7UkwOXcJOa9J6GQr7aA= MIME-Version: 1.0 Received: by 10.142.61.23 with SMTP id j23mr353891wfa.77.1267152646968; Thu, 25 Feb 2010 18:50:46 -0800 (PST) Date: Thu, 25 Feb 2010 18:50:46 -0800 Message-ID: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> From: Garrett Cooper To: FreeBSD-Hackers Content-Type: text/plain; charset=ISO-8859-1 Subject: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 02:50:52 -0000 Hi Hackers, Really basic question (because I'm relatively new to the Unix scene -- only been using it for the last 10 years, so I don't know if this was done for backwards compatibility with SysV) -- is mktemp(1) without -t supposed to default to $PWD instead of /tmp if a template is specified, e.g. [root@left4dead /usr/home/garrcoop]# mktemp fooXXXXXX foovE3FLt [root@left4dead /usr/home/garrcoop]# ls foovE3FLt foovE3FLt I ask because GNU coreutils' copy of mktemp (I know... I know...) defaults to /tmp if $TMPDIR isn't specified. Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 05:00:45 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DAFC11065670 for ; Fri, 26 Feb 2010 05:00:45 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id AD3F98FC26 for ; Fri, 26 Feb 2010 05:00:45 +0000 (UTC) Received: by pwj7 with SMTP id 7so7023870pwj.13 for ; Thu, 25 Feb 2010 21:00:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=/ew7VG1bgwY3SldrC/aO4i4Or6RYs80JmVfEUGJHBZs=; b=lh0EVKuSe0K2ct75lZNPHA1ouaB11+vRjFTmJ1cwjUi7Pgcj8QvXEunY7TyRMnemYp rG/guPuHq/3EjIvTWF1k2owC/TspzsTeqKz479Y8OIDynGbjbWmNHq6v2Q2MTLTXwlWH EtUsXFFmvwZXy7+9RNaMAeV0LNpVvexgnVz5Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=MXMXovEa6N2EZkpCKVFpG5/nRH2IScv5UX90ZTfUsqJ++tNcrYO3cNtbnVAWGdxIDx R1UUz8+v+s9/7aJU7Y1jlsNwGDLcMpY6Do5Kd6t4dleOIiKGek2nEB+KIJJkkql4jBDW Ovt6jYJl+vXdZMhbiRYEhSZ1lGA1XBGMBXaHo= MIME-Version: 1.0 Received: by 10.142.61.23 with SMTP id j23mr424536wfa.77.1267160442675; Thu, 25 Feb 2010 21:00:42 -0800 (PST) In-Reply-To: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> References: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> Date: Thu, 25 Feb 2010 21:00:42 -0800 Message-ID: <7d6fde3d1002252100oc64434ci5f6783ff10a9f0ea@mail.gmail.com> From: Garrett Cooper To: FreeBSD-Hackers Content-Type: multipart/mixed; boundary=001636e1f73d0f220b048079c89a Subject: Re: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 05:00:46 -0000 --001636e1f73d0f220b048079c89a Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Thu, Feb 25, 2010 at 6:50 PM, Garrett Cooper wrote: > Hi Hackers, > =A0 =A0Really basic question (because I'm relatively new to the Unix > scene -- only been using it for the last 10 years, so I don't know if > this was done for backwards compatibility with SysV) -- is mktemp(1) > without -t supposed to default to $PWD instead of /tmp if a template > is specified, e.g. > > [root@left4dead /usr/home/garrcoop]# mktemp fooXXXXXX > foovE3FLt > [root@left4dead /usr/home/garrcoop]# ls foovE3FLt > foovE3FLt > > =A0 =A0I ask because GNU coreutils' copy of mktemp (I know... I know...) > defaults to /tmp if $TMPDIR isn't specified. And this is the reason why I asked... I was getting annoyed when I ran out of space in /usr today iteratively trying to generate mfsroots because I expected the default temp directory to be /tmp. Turns out it wasn't, and the reason why is that $PWD is the assumed $TMPDIR iff an argument with -t isn't specified. So what I did was I wrote up a patch to be *I know... here it comes* more like GNU coreutils' copy of mktemp. Three behavioral changes I'm proposing are: 1. If mktemp is called without a prefix or template, it will automatically generate one file (or directory) instead of erroring out. That way if someone wants to call mktemp, they will automatically get a file instead of having to produce a template or prefix. 2. $TMPDIR (or /tmp if not specified) will automatically get tacked onto each file, instead of it being implicitly $PWD; that way a bunch of temporary files will be generated _in_ /tmp (and subsequently wiped on each reboot if the sysadmin sets it up that way) instead of having a boatload of temporary files spread across a machine, which the user must clean up based on the current working directory for their applications. 3. All files will be prefixed with `tmp' by default instead of `mktemp' (for consistency with GNU coreutils, and also because it allows folks to increase the size of the random string in the files). If folks like it, I'll create a PR with this change and also update the manpage to reflect the change. Thanks, -Garrett Index: mktemp.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- mktemp.c (revision 204344) +++ mktemp.c (working copy) @@ -48,6 +48,12 @@ static void usage(void); +#define MK_TEMPLATE(template) \ + asprintf(&name, "%s%s%s", \ + (tmpdir =3D=3D NULL ? _PATH_TMP : tmpdir), \ + (tmpdir =3D=3D NULL ? "" : "/" ), \ + template) + int main(int argc, char **argv) { @@ -55,13 +61,13 @@ char *tmpdir; const char *prefix; char *name; - int dflag, qflag, tflag, uflag; + int dflag, qflag, uflag; - ret =3D dflag =3D qflag =3D tflag =3D uflag =3D 0; - prefix =3D "mktemp"; + ret =3D dflag =3D qflag =3D uflag =3D 0; + prefix =3D NULL; name =3D NULL; - while ((c =3D getopt(argc, argv, "dqt:u")) !=3D -1) + while ((c =3D getopt(argc, argv, "dqt:u")) !=3D -1) { switch (c) { case 'd': dflag++; @@ -73,7 +79,6 @@ case 't': prefix =3D optarg; - tflag++; break; case 'u': @@ -83,16 +88,29 @@ default: usage(); } + } argc -=3D optind; argv +=3D optind; - if (tflag) { - tmpdir =3D getenv("TMPDIR"); - if (tmpdir =3D=3D NULL) - asprintf(&name, "%s%s.XXXXXXXX", _PATH_TMP, prefix); - else - asprintf(&name, "%s/%s.XXXXXXXX", tmpdir, prefix); + tmpdir =3D getenv("TMPDIR"); + + /* + * User didn't specify anything; let's default to a file prefixed by + * tmp* + */ + if (prefix =3D=3D NULL && argc =3D=3D 0) { + prefix =3D "tmp.XXXXXX"; + } + + /* + * Make sure that the user specified an option with -t (or nothing + * at all -- which equates to a file in tmp prefixed file). + */ + if (prefix !=3D NULL) { + + MK_TEMPLATE(prefix); + /* if this fails, the program is in big trouble already */ if (name =3D=3D NULL) { if (qflag) @@ -100,44 +118,58 @@ else errx(1, "cannot generate template"); } - } else if (argc < 1) { - usage(); + } - =09 + /* generate all requested files */ while (name !=3D NULL || argc > 0) { + if (name =3D=3D NULL) { - name =3D strdup(argv[0]); + MK_TEMPLATE(argv[0]); argv++; argc--; } - if (dflag) { - if (mkdtemp(name) =3D=3D NULL) { - ret =3D 1; - if (!qflag) - warn("mkdtemp failed on %s", name); + /* Just in case asprintf(3) ala MK_TEMPLATE fails. */ + if (name !=3D NULL) { + + if (dflag) { + + if (mkdtemp(name) =3D=3D NULL) { + ret =3D 1; + if (!qflag) { + warn("mkdtemp failed on %s", + name); + } + } else { + printf("%s\n", name); + if (uflag) + rmdir(name); + } } else { - printf("%s\n", name); - if (uflag) - rmdir(name); + + fd =3D mkstemp(name); + if (fd < 0) { + ret =3D 1; + if (!qflag) { + warn("mkstemp failed on %s", + name); + } + } else { + close(fd); + if (uflag) + unlink(name); + printf("%s\n", name); + } + } - } else { - fd =3D mkstemp(name); - if (fd < 0) { - ret =3D 1; - if (!qflag) - warn("mkstemp failed on %s", name); - } else { - close(fd); - if (uflag) - unlink(name); - printf("%s\n", name); - } + + free(name); + } - if (name) - free(name); + name =3D NULL; + } return (ret); } --001636e1f73d0f220b048079c89a Content-Type: application/octet-stream; name="more-gnuish-mktemp.diff" Content-Disposition: attachment; filename="more-gnuish-mktemp.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_g64ilrwc1 SW5kZXg6IG1rdGVtcC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIG1rdGVtcC5jCShyZXZpc2lvbiAyMDQzNDQp CisrKyBta3RlbXAuYwkod29ya2luZyBjb3B5KQpAQCAtNDgsNiArNDgsMTIgQEAKIAogc3RhdGlj IHZvaWQgdXNhZ2Uodm9pZCk7CiAKKyNkZWZpbmUgTUtfVEVNUExBVEUodGVtcGxhdGUpIFwKKwlh c3ByaW50ZigmbmFtZSwgIiVzJXMlcyIsIFwKKwkJCSh0bXBkaXIgPT0gTlVMTCA/IF9QQVRIX1RN UCA6IHRtcGRpciksIFwKKwkJCSh0bXBkaXIgPT0gTlVMTCA/ICIiIDogIi8iICksIFwKKwkJCXRl bXBsYXRlKQorCiBpbnQKIG1haW4oaW50IGFyZ2MsIGNoYXIgKiphcmd2KQogewpAQCAtNTUsMTMg KzYxLDEzIEBACiAJY2hhciAqdG1wZGlyOwogCWNvbnN0IGNoYXIgKnByZWZpeDsKIAljaGFyICpu YW1lOwotCWludCBkZmxhZywgcWZsYWcsIHRmbGFnLCB1ZmxhZzsKKwlpbnQgZGZsYWcsIHFmbGFn LCB1ZmxhZzsKIAotCXJldCA9IGRmbGFnID0gcWZsYWcgPSB0ZmxhZyA9IHVmbGFnID0gMDsKLQlw cmVmaXggPSAibWt0ZW1wIjsKKwlyZXQgPSBkZmxhZyA9IHFmbGFnID0gdWZsYWcgPSAwOworCXBy ZWZpeCA9IE5VTEw7CiAJbmFtZSA9IE5VTEw7CiAKLQl3aGlsZSAoKGMgPSBnZXRvcHQoYXJnYywg YXJndiwgImRxdDp1IikpICE9IC0xKQorCXdoaWxlICgoYyA9IGdldG9wdChhcmdjLCBhcmd2LCAi ZHF0OnUiKSkgIT0gLTEpIHsKIAkJc3dpdGNoIChjKSB7CiAJCWNhc2UgJ2QnOgogCQkJZGZsYWcr KzsKQEAgLTczLDcgKzc5LDYgQEAKIAogCQljYXNlICd0JzoKIAkJCXByZWZpeCA9IG9wdGFyZzsK LQkJCXRmbGFnKys7CiAJCQlicmVhazsKIAogCQljYXNlICd1JzoKQEAgLTgzLDE2ICs4OCwyOSBA QAogCQlkZWZhdWx0OgogCQkJdXNhZ2UoKTsKIAkJfQorCX0KIAogCWFyZ2MgLT0gb3B0aW5kOwog CWFyZ3YgKz0gb3B0aW5kOwogCi0JaWYgKHRmbGFnKSB7Ci0JCXRtcGRpciA9IGdldGVudigiVE1Q RElSIik7Ci0JCWlmICh0bXBkaXIgPT0gTlVMTCkKLQkJCWFzcHJpbnRmKCZuYW1lLCAiJXMlcy5Y WFhYWFhYWCIsIF9QQVRIX1RNUCwgcHJlZml4KTsKLQkJZWxzZQotCQkJYXNwcmludGYoJm5hbWUs ICIlcy8lcy5YWFhYWFhYWCIsIHRtcGRpciwgcHJlZml4KTsKKwl0bXBkaXIgPSBnZXRlbnYoIlRN UERJUiIpOworCisJLyogCisJICogVXNlciBkaWRuJ3Qgc3BlY2lmeSBhbnl0aGluZzsgbGV0J3Mg ZGVmYXVsdCB0byBhIGZpbGUgcHJlZml4ZWQgYnkKKwkgKiB0bXAqCisJICovCisJaWYgKHByZWZp eCA9PSBOVUxMICYmIGFyZ2MgPT0gMCkgeworCQlwcmVmaXggPSAidG1wLlhYWFhYWCI7CisJfQor CisJLyogCisJICogTWFrZSBzdXJlIHRoYXQgdGhlIHVzZXIgc3BlY2lmaWVkIGFuIG9wdGlvbiB3 aXRoIC10IChvciBub3RoaW5nCisJICogYXQgYWxsIC0tIHdoaWNoIGVxdWF0ZXMgdG8gYSBmaWxl IGluIHRtcCBwcmVmaXhlZCBmaWxlKS4KKwkgKi8KKwlpZiAocHJlZml4ICE9IE5VTEwpIHsKKwor CQlNS19URU1QTEFURShwcmVmaXgpOworCiAJCS8qIGlmIHRoaXMgZmFpbHMsIHRoZSBwcm9ncmFt IGlzIGluIGJpZyB0cm91YmxlIGFscmVhZHkgKi8KIAkJaWYgKG5hbWUgPT0gTlVMTCkgewogCQkJ aWYgKHFmbGFnKQpAQCAtMTAwLDQ0ICsxMTgsNTggQEAKIAkJCWVsc2UKIAkJCQllcnJ4KDEsICJj YW5ub3QgZ2VuZXJhdGUgdGVtcGxhdGUiKTsKIAkJfQotCX0gZWxzZSBpZiAoYXJnYyA8IDEpIHsK LQkJdXNhZ2UoKTsKKwogCX0KLQkJCisKIAkvKiBnZW5lcmF0ZSBhbGwgcmVxdWVzdGVkIGZpbGVz ICovCiAJd2hpbGUgKG5hbWUgIT0gTlVMTCB8fCBhcmdjID4gMCkgeworCiAJCWlmIChuYW1lID09 IE5VTEwpIHsKLQkJCW5hbWUgPSBzdHJkdXAoYXJndlswXSk7CisJCQlNS19URU1QTEFURShhcmd2 WzBdKTsKIAkJCWFyZ3YrKzsKIAkJCWFyZ2MtLTsKIAkJfQogCi0JCWlmIChkZmxhZykgewotCQkJ aWYgKG1rZHRlbXAobmFtZSkgPT0gTlVMTCkgewotCQkJCXJldCA9IDE7Ci0JCQkJaWYgKCFxZmxh ZykKLQkJCQkJd2FybigibWtkdGVtcCBmYWlsZWQgb24gJXMiLCBuYW1lKTsKKwkJLyogSnVzdCBp biBjYXNlIGFzcHJpbnRmKDMpIGFsYSBNS19URU1QTEFURSBmYWlscy4gKi8KKwkJaWYgKG5hbWUg IT0gTlVMTCkgeworCisJCQlpZiAoZGZsYWcpIHsKKworCQkJCWlmIChta2R0ZW1wKG5hbWUpID09 IE5VTEwpIHsKKwkJCQkJcmV0ID0gMTsKKwkJCQkJaWYgKCFxZmxhZykgeworCQkJCQkJd2Fybigi bWtkdGVtcCBmYWlsZWQgb24gJXMiLAorCQkJCQkJICAgICBuYW1lKTsKKwkJCQkJfQorCQkJCX0g ZWxzZSB7CisJCQkJCXByaW50ZigiJXNcbiIsIG5hbWUpOworCQkJCQlpZiAodWZsYWcpCisJCQkJ CQlybWRpcihuYW1lKTsKKwkJCQl9CiAJCQl9IGVsc2UgewotCQkJCXByaW50ZigiJXNcbiIsIG5h bWUpOwotCQkJCWlmICh1ZmxhZykKLQkJCQkJcm1kaXIobmFtZSk7CisKKwkJCQlmZCA9IG1rc3Rl bXAobmFtZSk7CisJCQkJaWYgKGZkIDwgMCkgeworCQkJCQlyZXQgPSAxOworCQkJCQlpZiAoIXFm bGFnKSB7CisJCQkJCQl3YXJuKCJta3N0ZW1wIGZhaWxlZCBvbiAlcyIsCisJCQkJCQkgICAgIG5h bWUpOworCQkJCQl9CisJCQkJfSBlbHNlIHsKKwkJCQkJY2xvc2UoZmQpOworCQkJCQlpZiAodWZs YWcpCisJCQkJCQl1bmxpbmsobmFtZSk7CisJCQkJCXByaW50ZigiJXNcbiIsIG5hbWUpOworCQkJ CX0KKwogCQkJfQotCQl9IGVsc2UgewotCQkJZmQgPSBta3N0ZW1wKG5hbWUpOwotCQkJaWYgKGZk IDwgMCkgewotCQkJCXJldCA9IDE7Ci0JCQkJaWYgKCFxZmxhZykKLQkJCQkJd2FybigibWtzdGVt cCBmYWlsZWQgb24gJXMiLCBuYW1lKTsKLQkJCX0gZWxzZSB7Ci0JCQkJY2xvc2UoZmQpOwotCQkJ CWlmICh1ZmxhZykKLQkJCQkJdW5saW5rKG5hbWUpOwotCQkJCXByaW50ZigiJXNcbiIsIG5hbWUp OwotCQkJfQorCisJCQlmcmVlKG5hbWUpOworCiAJCX0KLQkJaWYgKG5hbWUpCi0JCQlmcmVlKG5h bWUpOworCiAJCW5hbWUgPSBOVUxMOworCiAJfQogCXJldHVybiAocmV0KTsKIH0K --001636e1f73d0f220b048079c89a-- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 05:16:22 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDB72106564A for ; Fri, 26 Feb 2010 05:16:22 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-px0-f195.google.com (mail-px0-f195.google.com [209.85.216.195]) by mx1.freebsd.org (Postfix) with ESMTP id B964B8FC0A for ; Fri, 26 Feb 2010 05:16:22 +0000 (UTC) Received: by pxi33 with SMTP id 33so1175386pxi.14 for ; Thu, 25 Feb 2010 21:16:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=0unmf2CGxkWOWpwq2RUpUqhuySvKyhCA21I561QeAYY=; b=Gmp/F0+LcAnCK4xWPwwmzdqAFz8lKSQ5kQXO85ClflZiwv0IUyniOL2s8oXdIx/t9B JFgmT2EJpYPa7JDnSjhrEGTCnygWEg9LIdmi466igc5Hit8gZilYhlUbSartrWbJRMmb D7wHbbsHOTjBaS0sxSTMWa3BDKIDIteMhC2/8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=INiQqBu25xI7clRHPzBvyV6GcE/mIr1lfmaYAFYSc4nwoe31HloJ7FY6Ym2ypJljdV 3ehFsAhZn5lAXCnwRwWZzDqQPg9NNjxAjd83RoezhFzRElc2r93/WSf8i75E23OBsVef R7TkFCOU+2NPUdmGEYR6fYGUadDNG84xT58vU= MIME-Version: 1.0 Received: by 10.142.6.32 with SMTP id 32mr496985wff.6.1267161376182; Thu, 25 Feb 2010 21:16:16 -0800 (PST) Date: Thu, 25 Feb 2010 21:16:16 -0800 Message-ID: <7d6fde3d1002252116k55a8e350s5dcc0846cba07785@mail.gmail.com> From: Garrett Cooper To: FreeBSD-Hackers Content-Type: text/plain; charset=ISO-8859-1 Subject: ddb and dump devices X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 05:16:22 -0000 Hi again Hackers, Another question that popped up recently in my mind given some of the stability issues with RAID controllers is: what happens when you can't dump to a RAID or non-RAIDed PATA/SATA device? In particular, does it make sense to have the functionality where one could make the dump/panic operation fault resistant where the person could get a `freebie' and use an alternate device, like USB thumbdrives, tapedrives (not saying that'd be smart), or a network device to dump the data? I realize that not all cases are recoverable and there will be some coverage gaps still, but it'd be nice if we could fill in the gaps that do exist when stuff goes horribly south. This functionality would be helpful for my company at least (Ironport) as disk RAIDs sometimes fail and we don't have a means of writing back data, and we don't always have serial connectivity on our appliances and our customers typically don't in the field, s.t. we can use kgdb on the machine and get more useful tracebacks from panic conditions. Please let me know what you think and whether or not this is a worthwhile endeavor. Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 05:39:04 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E47B106566B for ; Fri, 26 Feb 2010 05:39:04 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ww0-f54.google.com (mail-ww0-f54.google.com [74.125.82.54]) by mx1.freebsd.org (Postfix) with ESMTP id CC80F8FC14 for ; Fri, 26 Feb 2010 05:39:03 +0000 (UTC) Received: by wwb22 with SMTP id 22so2677588wwb.13 for ; Thu, 25 Feb 2010 21:38:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=tEyCajGyEkk+9wndb7jrvdlthfzSRBnffMPVMJUeHiY=; b=tmBfpTgYzoQlfHaYMESqmMv1QDvrpfTwfiXEcVzCH83/TyhIk1gmZCWb4wpmaR2w0g KiMtqXAFJObNJBb5EQG5GjAeM6dJqXMyp/px1HzbcdqOo3c63OJf+0hKPHkaqC7/iSkk vPKBi3kMniJDmiHLQNX4a8uK/dEhEoINhIxEs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=VJa8/FTTEIgUpQHXLIBTPKN2MOKc1wf2ForDcf/D8stvx7u0bupfTYgotjwGquFK5F FBeOqTW+vcPDrZc0P4E8Yx+C5JYmofFQLu8LSuWJ0J1UBq8IDF+9/JlVKWbyL5O8XYOt Gc7fkeN7/BmAyKLAduDmisAeJUMt5eHg+TnNI= MIME-Version: 1.0 Received: by 10.216.163.67 with SMTP id z45mr1038116wek.26.1267162737908; Thu, 25 Feb 2010 21:38:57 -0800 (PST) In-Reply-To: <7d6fde3d1002252116k55a8e350s5dcc0846cba07785@mail.gmail.com> References: <7d6fde3d1002252116k55a8e350s5dcc0846cba07785@mail.gmail.com> Date: Fri, 26 Feb 2010 00:38:57 -0500 Message-ID: From: Ryan Stone To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD-Hackers Subject: Re: ddb and dump devices X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 05:39:04 -0000 I think a better idea would be to dump over the network to another server. We do that at $(WORK) for diskless machines and it's quite effective. Currently what we have is very rough and only implemented on 6.1, but if I get some time this weekend I'll try to clean it up and provide a patch against HEAD. Ryan Stone From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 06:27:44 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 996AF106566B for ; Fri, 26 Feb 2010 06:27:44 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id 285168FC17 for ; Fri, 26 Feb 2010 06:27:43 +0000 (UTC) Received: (qmail 11566 invoked by uid 399); 26 Feb 2010 06:27:43 -0000 Received: from localhost (HELO ?192.168.0.145?) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 26 Feb 2010 06:27:43 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Date: Thu, 25 Feb 2010 22:27:40 -0800 (PST) From: Doug Barton To: Garrett Cooper In-Reply-To: <7d6fde3d1002252100oc64434ci5f6783ff10a9f0ea@mail.gmail.com> Message-ID: References: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> <7d6fde3d1002252100oc64434ci5f6783ff10a9f0ea@mail.gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-message-flag: Outlook -- Not just for spreading viruses anymore! X-OpenPGP-Key-ID: 0xD5B2F0FB Organization: http://SupersetSolutions.com/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD-Hackers Subject: Re: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 06:27:44 -0000 On Thu, 25 Feb 2010, Garrett Cooper wrote: > So what I did was I wrote up a patch to be *I know... here it comes* > more like GNU coreutils' copy of mktemp. What's the motivation for this? I'm a little confused about why we'd want to change this when the -t option already exists. Also, does POSIX say anything about what the default should be? Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 09:21:27 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C8B8106564A for ; Fri, 26 Feb 2010 09:21:27 +0000 (UTC) (envelope-from erikt@midgard.homeip.net) Received: from ch-smtp02.sth.basefarm.net (ch-smtp02.sth.basefarm.net [80.76.149.213]) by mx1.freebsd.org (Postfix) with ESMTP id BA9528FC17 for ; Fri, 26 Feb 2010 09:21:26 +0000 (UTC) Received: from c83-255-48-78.bredband.comhem.se ([83.255.48.78]:49611 helo=falcon.midgard.homeip.net) by ch-smtp02.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1NkwNu-00083p-7c for freebsd-hackers@freebsd.org; Fri, 26 Feb 2010 10:21:24 +0100 Received: (qmail 38469 invoked from network); 26 Feb 2010 10:21:19 +0100 Received: from owl.midgard.homeip.net (10.1.5.7) by falcon.midgard.homeip.net with ESMTP; 26 Feb 2010 10:21:19 +0100 Received: (qmail 61645 invoked by uid 1001); 26 Feb 2010 10:21:19 +0100 Date: Fri, 26 Feb 2010 10:21:19 +0100 From: Erik Trulsson To: Doug Barton Message-ID: <20100226092119.GA61498@owl.midgard.homeip.net> References: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> <7d6fde3d1002252100oc64434ci5f6783ff10a9f0ea@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-Originating-IP: 83.255.48.78 X-Scan-Result: No virus found in message 1NkwNu-00083p-7c. X-Scan-Signature: ch-smtp02.sth.basefarm.net 1NkwNu-00083p-7c c2523862898c9e249e89c73e35e025f8 Cc: Garrett Cooper , FreeBSD-Hackers Subject: Re: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 09:21:27 -0000 On Thu, Feb 25, 2010 at 10:27:40PM -0800, Doug Barton wrote: > On Thu, 25 Feb 2010, Garrett Cooper wrote: > > > So what I did was I wrote up a patch to be *I know... here it comes* > > more like GNU coreutils' copy of mktemp. > > What's the motivation for this? I'm a little confused about why we'd > want to change this when the -t option already exists. Also, does POSIX > say anything about what the default should be? POSIX does not define the mktemp(1) utility as far as I can tell, and thus says nothing about the default. The HISTORY section in the manpage says that mktemp(1) originated with OpenBSD so if anything it is the OpenBSD implementation that ought to be used as a reference. If the GNU implementation behaves differently, then I would say it is likely the GNU version which is wrong. -- Erik Trulsson ertr1013@student.uu.se From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 09:47:17 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10248106566C; Fri, 26 Feb 2010 09:47:17 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id D38DE8FC13; Fri, 26 Feb 2010 09:47:16 +0000 (UTC) Received: by pwj7 with SMTP id 7so7169544pwj.13 for ; Fri, 26 Feb 2010 01:47:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=Lcek5CXYxth37dUqB52SxLjN/fBiI587lzGZaBUNrZI=; b=EBQn0GaDVvx5+OeiYrrGVDpzK51jG0o41RRPR+fCUr0ZXV4GptcvyUVCs+avB6Z8KT 7+UxNAys3w7ImEUA9ONKhL3LBUTxsUpdcQkVTYlnX4m42HqlRe8/mMJdCZ7ppVmn9z0L VOEispQtbJSIk0l5ClJcmhebqKwH3ny4TqLzk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Ig+nFFRfcNY8DgNFnCifyR7n3CaJUPWUx5k6NR67uTJdix6hbkLZxEmFCks6OHT0wl k+tWlghW3q8lDhj0JLqeXsnbqw0OXBV4yTSmr+lH1nLWm3KpIyQ/41ICI7A8zzcuRkLD DhveFS11yTFsTL2nTaDgenglY3qq7YzAtpggM= MIME-Version: 1.0 Received: by 10.142.195.18 with SMTP id s18mr56390wff.325.1267177631423; Fri, 26 Feb 2010 01:47:11 -0800 (PST) In-Reply-To: References: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> <7d6fde3d1002252100oc64434ci5f6783ff10a9f0ea@mail.gmail.com> Date: Fri, 26 Feb 2010 01:47:11 -0800 Message-ID: <7d6fde3d1002260147h428dc97ere1722e8b658d321e@mail.gmail.com> From: Garrett Cooper To: Doug Barton Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD-Hackers Subject: Re: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 09:47:17 -0000 On Thu, Feb 25, 2010 at 10:27 PM, Doug Barton wrote: > On Thu, 25 Feb 2010, Garrett Cooper wrote: > >> So what I did was I wrote up a patch to be *I know... here it comes* >> more like GNU coreutils' copy of mktemp. > > What's the motivation for this? Less typing and quirks because every other command line and programmatic implementation I've run across (Linux, perl, python) uses a similar paradigm with these sets of APIs. Example: gcooper@orangebox ~/Downloads/susv4/susv4 $ uname -or 2.6.31-gentoo-r6 GNU/Linux gcooper@orangebox ~/Downloads/susv4/susv4 $ mktemp /tmp/tmp.5pmHxn3UfQ gcooper@orangebox ~/Downloads/susv4/susv4 $ mktemp -d /tmp/tmp.Q0PZbsf9BG Compared with: [gcooper@optimus ~]$ mktemp usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix [gcooper@optimus ~]$ mktemp -d usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix [gcooper@optimus ~]$ mktemp -d tmp.XXXXXXXXXX tmp.cvZo2ABNRv [gcooper@optimus ~]$ mktemp tmp.XXXXXXXXXX tmp.KFnF5zomVn > I'm a little confused about why we'd want to change this when the -t option already exists. Because 1) -t requires that you spell out the entire path in a more terse way, 2) tacks on more information in a more counter intuitive manner, and 3) can only be used once per mktemp(3) call. Example: [gcooper@optimus ~]$ mktemp -t tmp /tmp/tmp.vAi6eCpD [gcooper@optimus ~]$ mktemp -t tmp -d /tmp/tmp.0f3Z9L5R [gcooper@optimus ~]$ mktemp -t tmp -t notsotmp -d /tmp/notsotmp.MTYHIGFs [gcooper@optimus ~]$ mktemp -t foo.XXXXXX /tmp/foo.XXXXXX.rGssqAZk [gcooper@optimus ~]$ mktemp -t /something/foo.XXXXXX mktemp: mkstemp failed on /tmp//something/foo.XXXXXX.J3CGs9dK: No such file or directory > Also, does POSIX say anything about what the default should be? Nope. No mention of mktemp(1) in either v3 or v4 of the POSIX 1003.1 specs. FWIW, the major advantages that BSD has on the coreutil's mktemp is that it's half the size of coreutil's mktemp; plus, you only need to call mktemp(1) once for multiple files. I have no intent on changing that behavior because I think it's cool and more innovative than the dumbed down single invocation mktemp(1) setup with coreutil's mktemp. Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 10:03:47 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A8AD8106566B; Fri, 26 Feb 2010 10:03:47 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pz0-f199.google.com (mail-pz0-f199.google.com [209.85.222.199]) by mx1.freebsd.org (Postfix) with ESMTP id 76BA78FC1E; Fri, 26 Feb 2010 10:03:47 +0000 (UTC) Received: by pzk37 with SMTP id 37so2180325pzk.7 for ; Fri, 26 Feb 2010 02:03:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=O0xWOGfRpIsGooQt7o5qOEdu31FE2yK+xgvsr9RsYNI=; b=hsnVE4uLvBv4M1PSUnm+5bI2gEL4Rt2TzeiM6YQcfO12eJIfg6oBrgdxQ+j0bz92IY 5UOyr8++4DSPgfeXro3dmxW2424u3l/R22zrfV9EfUDQ+UWFn1m3UwgxhtrS1/99PHzU rrENdcs6QmL8L29u58T10YTu+s7xKlfX1gRAg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=pY+wJn9mEhU9Zj6QB6KkbUDg6XeY3Za5UDP33muqDg0le1NpxcBuOxQHi1bCzsPAar 7zHYz9uNQ3HSohw928Q1mbX2gvTZlm43D3NJqOEQs2+5upGlSSf7hb6pGBfmkJrqJB/a 5vN/mf5eOqRXlddkoc3ucUX7gEcWrPlY7u4kQ= MIME-Version: 1.0 Received: by 10.142.151.15 with SMTP id y15mr67747wfd.310.1267178622561; Fri, 26 Feb 2010 02:03:42 -0800 (PST) In-Reply-To: <20100226092119.GA61498@owl.midgard.homeip.net> References: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> <7d6fde3d1002252100oc64434ci5f6783ff10a9f0ea@mail.gmail.com> <20100226092119.GA61498@owl.midgard.homeip.net> Date: Fri, 26 Feb 2010 02:03:42 -0800 Message-ID: <7d6fde3d1002260203l5c7491c7w9ed84cdf40acf9d7@mail.gmail.com> From: Garrett Cooper To: Erik Trulsson Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD-Hackers , Doug Barton Subject: Re: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 10:03:47 -0000 On Fri, Feb 26, 2010 at 1:21 AM, Erik Trulsson wrote: > On Thu, Feb 25, 2010 at 10:27:40PM -0800, Doug Barton wrote: >> On Thu, 25 Feb 2010, Garrett Cooper wrote: >> >> > So what I did was I wrote up a patch to be *I know... here it comes* >> > more like GNU coreutils' copy of mktemp. >> >> What's the motivation for this? I'm a little confused about why we'd >> want to change this when the -t option already exists. Also, does POSIX >> say anything about what the default should be? > > POSIX does not define the mktemp(1) utility as far as I can tell, and > thus says nothing about the default. > > The HISTORY section in the manpage says that mktemp(1) originated with > OpenBSD so if anything it is the OpenBSD implementation that ought to > be used as a reference. > > If the GNU implementation behaves differently, then I would say it is > likely the GNU version which is wrong. I'm not going to get into that bikeshed topic. I'm not arguing about what's right or wrong -- I just prefer not dealing with quirks between different systems, like having to type `find .' instead of just `find', which searches $PWD first with coreutil's find, or having to type the entry in fstab exactly when doing a mount or unmount because the directory isn't properly abspath'ed. FreeBSD is a great system; if there are ways that I can possibly make it better by adding smart defaults I will propose them wherever I possibly can, because if I've thought of something, I'm pretty sure I'm not the first one to have thought of it. Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 10:29:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89EF21065670 for ; Fri, 26 Feb 2010 10:29:36 +0000 (UTC) (envelope-from j.mckeown@ru.ac.za) Received: from f.mail.ru.ac.za (f.mail.ru.ac.za [IPv6:2001:4200:1010::25:6]) by mx1.freebsd.org (Postfix) with ESMTP id 98F248FC15 for ; Fri, 26 Feb 2010 10:29:35 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=ru-msa; d=ru.ac.za; h=Received:From:Organization:To:Subject:Date:User-Agent:References:In-Reply-To:X-Face:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-Disposition:Message-Id:X-Virus-Scanned:X-Authenticated-User; b=MVfBrYijFHY1p9J27oZa4q9/PnvJ4vTXY+XW6CJXZMdsuOWOnPEhiKggH4PjY5CFPhuKk49L4cTzhZOc8aMyLGshKdMHwYPbJHYhttlURRVF6VKDROwoZwjgRGLypIL8; Received: from vorkosigan.ru.ac.za ([2001:4200:1010:1058:219:d1ff:fe9f:a932]:54229) by f.mail.ru.ac.za with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1NkxRW-000Ee7-IJ for freebsd-hackers@freebsd.org; Fri, 26 Feb 2010 12:29:10 +0200 From: Jonathan McKeown Organization: Rhodes University To: freebsd-hackers@freebsd.org Date: Fri, 26 Feb 2010 12:29:10 +0200 User-Agent: KMail/1.9.10 References: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> <20100226092119.GA61498@owl.midgard.homeip.net> <7d6fde3d1002260203l5c7491c7w9ed84cdf40acf9d7@mail.gmail.com> In-Reply-To: <7d6fde3d1002260203l5c7491c7w9ed84cdf40acf9d7@mail.gmail.com> X-Face: $@VrUx^RHy/}yu]jKf/<4T%/d|F+$j-Ol2"2J$q+%OK1]&/G_S9(=?utf-8?q?HkaQ*=60!=3FYOK=3FY!=27M=60C=0A=09aP=5C9nVPF8Q=7DCilHH8l=3B=7E!4?= =?utf-8?q?2HK6=273lg4J=7Daz?=@1Dqqh:J]M^"YPn*2IWrZON$1+G?oX3@ =?utf-8?q?k=230=0A=0954XDRg=3DYn=5FF-etwot4U=24b?=dTS{i X-Virus-Scanned: f.mail.ru.ac.za (2001:4200:1010::25:6) X-Authenticated-User: s0900137 from vorkosigan.ru.ac.za (2001:4200:1010:1058:219:d1ff:fe9f:a932) using auth_plaintext Subject: Re: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 10:29:36 -0000 On Friday 26 February 2010 12:03:42 Garrett Cooper wrote: > FreeBSD is a great system; if there are ways that I can possibly make > it better by adding smart defaults Be careful about that value judgement. There are certainly ways you can change it. Not everyone might feel the change is for the better, and changing the default behaviour of widely-used commands is one change that you might find some people disagree with. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 10:56:34 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06105106566C for ; Fri, 26 Feb 2010 10:56:34 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id CF4618FC0C for ; Fri, 26 Feb 2010 10:56:33 +0000 (UTC) Received: by pvg3 with SMTP id 3so1721143pvg.13 for ; Fri, 26 Feb 2010 02:56:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=2/nBEBU+e8rygodJd0JI7U3yTDqI8a/wiLCBAzMKtcM=; b=MRpmnvEiOiYd9Di0+0xPN8fiijF8mfYxSP/ewze5BuaTTtbnl3ddcoHNKFCBhFfcqi bKFnWJlA3wNC9DOObQ/i2XjmecUJ7B3FPDpSt+xOg2nM6b7Vt93JAi/2ssEuEUo6VcjK 1mPsJB6gK7plozgRWm1++PdwYqiLb9014kYd4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=g+Z/2DTjuTy/CPxcn873RXYOasUHa/fLicd4ZaaIsmVuTWV9XH1IaIIsop3B1gvR3K qkNmiE0G33w/SEVx7Znh7014zPxeTddMQGWBWxPik6RGMFSxYRcgbOMov6lnVg/WNNIX wiyzV2JHw1hOAirRJ9LIDAaOt63/aM/MqWSyI= MIME-Version: 1.0 Received: by 10.142.248.18 with SMTP id v18mr121922wfh.97.1267181783796; Fri, 26 Feb 2010 02:56:23 -0800 (PST) In-Reply-To: <201002261229.10310.j.mckeown@ru.ac.za> References: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> <20100226092119.GA61498@owl.midgard.homeip.net> <7d6fde3d1002260203l5c7491c7w9ed84cdf40acf9d7@mail.gmail.com> <201002261229.10310.j.mckeown@ru.ac.za> Date: Fri, 26 Feb 2010 02:56:23 -0800 Message-ID: <7d6fde3d1002260256g71b6e431i705cb8180fdd0a5d@mail.gmail.com> From: Garrett Cooper To: Jonathan McKeown Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 10:56:34 -0000 On Fri, Feb 26, 2010 at 2:29 AM, Jonathan McKeown wrote: > On Friday 26 February 2010 12:03:42 Garrett Cooper wrote: >> FreeBSD is a great system; if there are ways that I can possibly make >> it better by adding smart defaults > > Be careful about that value judgement. There are certainly ways you can change > it. Not everyone might feel the change is for the better, and changing the > default behaviour of widely-used commands is one change that you might find > some people disagree with. I agree that there are some things that I'm going to have to apply on my own personal machines to achieve behavior that I want, and I agree that my way may not be the best way to do things, but that's part of the reasoning why I'm asking whether or not others find some value in what I'm proposing... Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 11:00:16 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96C611065679 for ; Fri, 26 Feb 2010 11:00:16 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-px0-f195.google.com (mail-px0-f195.google.com [209.85.216.195]) by mx1.freebsd.org (Postfix) with ESMTP id 6BBB58FC20 for ; Fri, 26 Feb 2010 11:00:16 +0000 (UTC) Received: by pxi33 with SMTP id 33so1261562pxi.14 for ; Fri, 26 Feb 2010 03:00:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=tcjn0CQ3L1lBwfUrO0QvRGI0gEwLTtFranPQjak3Q3Y=; b=pZVJ4x2CRdWbZ3H8Kf4jC4GWZE+LyFmieygrx1yI0R0LFXUzqXoLJyBxYvdNwgVMOC AE0yYXjbCycyTfC/PikwZMKMGBI0w18vKGMRBap6H11GgZ6d1dB+TpOdqXnyuFpwgyRv /M2nT+cr13WJ/SdQHC6wgo41t8SxtzJNhhc+I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=KxuPWNVrAILPpwrTWytvhuVq9m3pNJ3KzGJ5dz/ugE4mZRBEWmY07gKNwVJoSJkxE/ YDCicpSkyl5kg04bIBoI51h6bHvzDXodJSfyp463hVCSF19IITkkNLt81+A6f8/rJU/W z5tUBvk/KIRihup6VZ2mtl8Ey69I8cNtOmQeE= MIME-Version: 1.0 Received: by 10.142.247.33 with SMTP id u33mr110431wfh.219.1267182008062; Fri, 26 Feb 2010 03:00:08 -0800 (PST) In-Reply-To: References: <7d6fde3d1002252116k55a8e350s5dcc0846cba07785@mail.gmail.com> Date: Fri, 26 Feb 2010 03:00:08 -0800 Message-ID: <7d6fde3d1002260300v36c420d4r79a18efe02c0d8ab@mail.gmail.com> From: Garrett Cooper To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD-Hackers Subject: Re: ddb and dump devices X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 11:00:16 -0000 On Thu, Feb 25, 2010 at 9:38 PM, Ryan Stone wrote: > I think a better idea would be to dump over the network to another > server. =A0We do that at $(WORK) for diskless machines and it's quite > effective. =A0Currently what we have is very rough and only implemented > on 6.1, but if I get some time this weekend I'll try to clean it up > and provide a patch against HEAD. Network dumping is definitely desired and should perhaps be the primary focus, but there are situations I could see where networking isn't possible for infrastructure or policy reasons. How does your group configure the system to dump to network shares (I presume nfs)? Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 11:23:39 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35B4E106566B; Fri, 26 Feb 2010 11:23:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 92AF88FC1C; Fri, 26 Feb 2010 11:23:38 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1QBNbC0064844 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Feb 2010 13:23:37 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o1QBNaw0056146; Fri, 26 Feb 2010 13:23:36 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o1QBNaJk056145; Fri, 26 Feb 2010 13:23:36 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 26 Feb 2010 13:23:36 +0200 From: Kostik Belousov To: Peter Steele Message-ID: <20100226112336.GF2489@deviant.kiev.zoral.com.ua> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225125838.13cd6c0e.ray@dlink.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> <20100225151828.GA2489@deviant.kiev.zoral.com.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95703@MBX03.exg5.exghost.com> <20100225164514.GB2489@deviant.kiev.zoral.com.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95765@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95991@MBX03.exg5.exghost.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XIiC+We3v3zHqZ6Z" Content-Disposition: inline In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95991@MBX03.exg5.exghost.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean Cc: "freebsd-hackers@freebsd.org" , freebsd-python@freebsd.org Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 11:23:39 -0000 --XIiC+We3v3zHqZ6Z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Feb 25, 2010 at 04:26:22PM -0600, Peter Steele wrote: > > We'll likely go with this solution instead of downgrading Python and th= e related libraries. >=20 > In fact I came up with another solution. I realized that since the proble= m was related to the process signal mask, instead of called ntpd directly, = wrap it up in a C app that resets the signal mask to something that works. = I have the following code: >=20 > sigset_t set, oset; > sigemptyset(&set); > pthread_sigmask(SIG_SETMASK, &set, &oset); > system("/usr/sbin/ntpd -g -q"); > pthread_sigmask(SIG_SETMASK, &oset, NULL); >=20 > I wrapped this up into a standalone app and call this from Python instead= of calling ntpd directly. This solved the problem--no more hang. Thanks ve= ry much to Kostik Belousov for his "wild guess" that this was related to th= e process signal mask. His guess was dead on. So this is arguably a Python bug. Did you contacted anybody who cares about the Python ? --XIiC+We3v3zHqZ6Z Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkuHrzgACgkQC3+MBN1Mb4jFJwCcDNnuSfbIUExMzREKJOkYiCXz jZsAn2H6LQChB1guq5NaUoZkKzeBsfiO =v7X+ -----END PGP SIGNATURE----- --XIiC+We3v3zHqZ6Z-- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 13:33:32 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9667A106566B for ; Fri, 26 Feb 2010 13:33:32 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505a.appriver.com [98.129.35.4]) by mx1.freebsd.org (Postfix) with ESMTP id 47A3E8FC08 for ; Fri, 26 Feb 2010 13:33:31 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Policy: GLOBAL - maxiscale.com X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.15 X-Note-Reverse-DNS: ht02.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.15] (HELO ht02.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 29691600; Fri, 26 Feb 2010 07:33:32 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by ht02.exg5.exghost.com ([98.129.23.15]) with mapi; Fri, 26 Feb 2010 07:33:31 -0600 From: Peter Steele To: Kostik Belousov Date: Fri, 26 Feb 2010 07:33:29 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acq21ilp6fSHXbwASta/5ZqqCcYv/gAEPG7w Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95A7A@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> <20100224190035.GA5026@wep4035.physik.uni-wuerzburg.de> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95522@MBX03.exg5.exghost.com> <20100225125838.13cd6c0e.ray@dlink.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95689@MBX03.exg5.exghost.com> <20100225151828.GA2489@deviant.kiev.zoral.com.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95703@MBX03.exg5.exghost.com> <20100225164514.GB2489@deviant.kiev.zoral.com.ua> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95765@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95991@MBX03.exg5.exghost.com> <20100226112336.GF2489@deviant.kiev.zoral.com.ua> In-Reply-To: <20100226112336.GF2489@deviant.kiev.zoral.com.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-hackers@freebsd.org" , "freebsd-python@freebsd.org" Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 13:33:32 -0000 >So this is arguably a Python bug. Did you contacted anybody who cares abou= t the Python ? I did not, mainly because this link: http://bugs.python.org/msg61870 seems to imply they are already aware of the problem. I agree it must be a = Python bug though. It worked in 2.5.1 but not in 2.5.5 and later, so clearl= y they changed how processes are launched from threads that has led to this= problem. One should not have to be forced to make explicit calls to change= the signal mask in order to launch an external app. Granted, we've only ha= d this issue with ntpd--other apps launch fine--but there is clearly someth= ing wrong somewhere for even one app to hang when it is spawned as a thread= . From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 14:50:25 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CBB6106564A for ; Fri, 26 Feb 2010 14:50:25 +0000 (UTC) (envelope-from shuvaev@physik.uni-wuerzburg.de) Received: from mailrelay.rz.uni-wuerzburg.de (mailrelay.rz.uni-wuerzburg.de [132.187.3.28]) by mx1.freebsd.org (Postfix) with ESMTP id C9A788FC1C for ; Fri, 26 Feb 2010 14:50:24 +0000 (UTC) Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 93DDE73478; Fri, 26 Feb 2010 15:50:20 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 90CC155707; Fri, 26 Feb 2010 15:50:20 +0100 (CET) X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de Received: from mail.physik.uni-wuerzburg.de (wthp192.physik.uni-wuerzburg.de [132.187.40.192]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 6A38D5D857; Fri, 26 Feb 2010 15:50:20 +0100 (CET) Received: from wep4035 ([132.187.37.35]) by mail.physik.uni-wuerzburg.de (Lotus Domino Release 8.5.1FP1) with ESMTP id 2010022615501883-71342 ; Fri, 26 Feb 2010 15:50:18 +0100 Received: by wep4035 (sSMTP sendmail emulation); Fri, 26 Feb 2010 15:50:19 +0100 Date: Fri, 26 Feb 2010 15:50:19 +0100 From: Alexey Shuvaev To: Garrett Cooper Message-ID: <20100226145019.GA93856@wep4035.physik.uni-wuerzburg.de> References: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> <7d6fde3d1002252100oc64434ci5f6783ff10a9f0ea@mail.gmail.com> <7d6fde3d1002260147h428dc97ere1722e8b658d321e@mail.gmail.com> MIME-Version: 1.0 In-Reply-To: <7d6fde3d1002260147h428dc97ere1722e8b658d321e@mail.gmail.com> Organization: Universitaet Wuerzburg User-Agent: Mutt/1.5.20 (2009-06-14) X-MIMETrack: Itemize by SMTP Server on domino1/uni-wuerzburg(Release 8.5.1FP1|January 05, 2010) at 02/26/2010 03:50:19 PM, Serialize by Router on domino1/uni-wuerzburg(Release 8.5.1FP1|January 05, 2010) at 02/26/2010 03:50:19 PM, Serialize complete at 02/26/2010 03:50:19 PM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Cc: FreeBSD-Hackers Subject: Re: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 14:50:25 -0000 On Fri, Feb 26, 2010 at 01:47:11AM -0800, Garrett Cooper wrote: > On Thu, Feb 25, 2010 at 10:27 PM, Doug Barton wrote: > > On Thu, 25 Feb 2010, Garrett Cooper wrote: > > > >> So what I did was I wrote up a patch to be *I know... here it comes* > >> more like GNU coreutils' copy of mktemp. > > > > What's the motivation for this? > > Less typing and quirks because every other command line and > programmatic implementation I've run across (Linux, perl, python) uses > a similar paradigm with these sets of APIs. Example: > > gcooper@orangebox ~/Downloads/susv4/susv4 $ uname -or > 2.6.31-gentoo-r6 GNU/Linux > gcooper@orangebox ~/Downloads/susv4/susv4 $ mktemp > /tmp/tmp.5pmHxn3UfQ > gcooper@orangebox ~/Downloads/susv4/susv4 $ mktemp -d > /tmp/tmp.Q0PZbsf9BG > > Compared with: > > [gcooper@optimus ~]$ mktemp > usage: mktemp [-d] [-q] [-t prefix] [-u] template ... > mktemp [-d] [-q] [-u] -t prefix > This likely to go in the bikeshed direction, but I can see arguments that it is expected behavior because (by typing just 'mktemp') you haven't specified what you want to achieve. Moreover I would say that ~> mktemp app1XXXX app1D7yv ~> ls app1D7yv films_list mbox obj tmp bin install mc34063ti.pdf papers to_do.tar data layer meep_todo pentomino doc libctl mgtkdatabox src drawings libctl.orig mnt svn is also the expected behavior. I would be surprised to find app1D7yv in /tmp/ or whatever else and not in PWD. That being said I think nobody uses mktemp(1) from the interactive prompt. But on this route what is wrong with explicit calls like 'mktemp /tmp/tmp.XXXXXXXX' in your shell script? On the general note relying on the specific defaults is the best way to seed random incompatibility into any program... > [gcooper@optimus ~]$ mktemp -d > usage: mktemp [-d] [-q] [-t prefix] [-u] template ... > mktemp [-d] [-q] [-u] -t prefix > [gcooper@optimus ~]$ mktemp -d tmp.XXXXXXXXXX > tmp.cvZo2ABNRv > [gcooper@optimus ~]$ mktemp tmp.XXXXXXXXXX > tmp.KFnF5zomVn > > > I'm a little confused about why we'd want to change this when the -t option already exists. > > Because 1) -t requires that you spell out the entire path in a more > terse way, 2) tacks on more information in a more counter intuitive > manner, and 3) can only be used once per mktemp(3) call. Example: > > [gcooper@optimus ~]$ mktemp -t tmp > /tmp/tmp.vAi6eCpD > [gcooper@optimus ~]$ mktemp -t tmp -d > /tmp/tmp.0f3Z9L5R > [gcooper@optimus ~]$ mktemp -t tmp -t notsotmp -d > /tmp/notsotmp.MTYHIGFs > [gcooper@optimus ~]$ mktemp -t foo.XXXXXX > /tmp/foo.XXXXXX.rGssqAZk > [gcooper@optimus ~]$ mktemp -t /something/foo.XXXXXX > mktemp: mkstemp failed on /tmp//something/foo.XXXXXX.J3CGs9dK: No such > file or directory > > > Also, does POSIX say anything about what the default should be? > > Nope. No mention of mktemp(1) in either v3 or v4 of the POSIX 1003.1 specs. > > FWIW, the major advantages that BSD has on the coreutil's mktemp is > that it's half the size of coreutil's mktemp; plus, you only need to > call mktemp(1) once for multiple files. I have no intent on changing > that behavior because I think it's cool and more innovative than the > dumbed down single invocation mktemp(1) setup with coreutil's mktemp. > 0.00$, Alexey. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 16:32:39 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 556FD1065674 for ; Fri, 26 Feb 2010 16:32:39 +0000 (UTC) (envelope-from xorquewasp@googlemail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id D82EF8FC19 for ; Fri, 26 Feb 2010 16:32:38 +0000 (UTC) Received: by wyb40 with SMTP id 40so145514wyb.13 for ; Fri, 26 Feb 2010 08:32:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:received:received :x-authentication-warning:date:from:to:subject:message-id :mime-version:content-type:content-disposition :content-transfer-encoding; bh=K01ihMwkteMc+yJ/rlj31nCcG1jbQZWdHK9YRt32m6Q=; b=fTaOrzBvsZpac7ihzUq1+P7w7tgUBCn31A02BwWUjeqGnA9MWvQIKcVRtQ669CdC+c CaBMZfHN9xDUY3UPZGRzMXCMZI/HCBdc3CFqJ3YPaR9/zSCtNRRUNEwS8Rgd8cD4GgPG Unbr7O1uWLgNEV8h18GnWHCyTB1xHV9Bccw1Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=x-authentication-warning:date:from:to:subject:message-id :mime-version:content-type:content-disposition :content-transfer-encoding; b=u8jtOpaN3wNCVMrM9SzPP43iHfRkn/7S4bYu1eUom53lNJtJVVfC0RRAcGI+bmABog YukImnPXMlHEcvjzCIkN9Z2GExV3GVpbZgsUJLR3tmPAyM1W7aj/t4l//vb3mLZBaTfN 0fHB1c38fCUaL6YXMFxtEaQlhiSs1cwTuY2DM= Received: by 10.216.89.138 with SMTP id c10mr568068wef.47.1267201951381; Fri, 26 Feb 2010 08:32:31 -0800 (PST) Received: from viper.internal.network ([78.143.196.85]) by mx.google.com with ESMTPS id g9sm849519gvc.18.2010.02.26.08.32.29 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 26 Feb 2010 08:32:30 -0800 (PST) Received: from viper.internal.network (localhost [127.0.0.1]) by viper.internal.network (Postfix) with ESMTP id 696EB4AC06 for ; Fri, 26 Feb 2010 16:32:28 +0000 (UTC) Received: (from m0@localhost) by viper.internal.network (8.14.3/8.14.3/Submit) id o1QGWRWG051021 for freebsd-hackers@freebsd.org; Fri, 26 Feb 2010 16:32:27 GMT (envelope-from xorquewasp@googlemail.com) X-Authentication-Warning: viper.internal.network: m0 set sender to xorquewasp@googlemail.com using -f Date: Fri, 26 Feb 2010 16:32:27 +0000 From: xorquewasp@googlemail.com To: freebsd-hackers@freebsd.org Message-ID: <20100226163227.GA15162@logik.internal.network> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Subject: package building failure irritation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 16:32:39 -0000 'Lo. I've come up against an infuriating problem with ports. I don't even know what causes it, let alone how to solve it. Essentially, to stop already-installed software from polluting port builds, I build ports in a jail. I put a list of ports in a file and use a trivial shell script to build them one after the other with: cd $port make clean make deinstall make package-recursive The jail is set up with: /etc/make.conf: WRKDIRPREFIX=3D /work DISTDIR=3D /distfiles PACKAGES=3D /pkg PORTSDIR=3D /ports With /ports, /distfiles and /pkg being mounted from outside the jail using nullfs (just because those directories exist on a ZFS array outside of the jail for performance reasons). Before building a set of packages, the jail is wiped clean with 'pkg_delete -a'. The problem is that it seems most ports just outright fail to produce packages. I log each build and often the last lines of the log files read something like: xchm: tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directo= ry tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or dire= ctory tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or direc= tory tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or direc= tory tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or dire= ctory tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or direc= tory tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or direc= tory tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or direc= tory tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or direc= tory tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or direc= tory tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or direc= tory tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or dire= ctory tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or dire= ctory tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 Creating package /pkg/All/docbook-4.2.tbz Registering depends: iso8879-1986_2 xmlcatmgr-2.2. Creating bzip'd tar ball in '/pkg/All/docbook-4.2.tbz' *** Error code 1 Stop in /ports/textproc/docbook-420. *** Error code 1 Stop in /ports/textproc/docbook-420. *** Error code 1 Stop in /ports/deskutils/xchm. fatal: failed gimp: =3D=3D=3D> Patching for libopenraw-0.0.8_2 =3D=3D=3D> Applying FreeBSD patches for libopenraw-0.0.8_2 =3D=3D=3D> libopenraw-0.0.8_2 depends on file: /usr/local/bin/intltool-ex= tract - found =3D=3D=3D> libopenraw-0.0.8_2 depends on executable: pkg-config - found =3D=3D=3D> libopenraw-0.0.8_2 depends on shared library: boost_thread.4 -= not found =3D=3D=3D> Verifying install for boost_thread.4 in /ports/devel/boost-li= bs =3D=3D=3D> Returning to build of libopenraw-0.0.8_2 Error: shared library "boost_thread.4" does not exist *** Error code 1 Stop in /ports/graphics/libopenraw. *** Error code 1 Stop in /ports/graphics/gegl. *** Error code 1 Stop in /ports/graphics/gimp-app. *** Error code 1 Stop in /ports/graphics/gimp. fatal: failed inkscape: Creating bzip'd tar ball in '/pkg/All/atk-1.28.0.tbz' =3D=3D=3D> Generating temporary packing list tar: lib/gtk-2.0/2.10.0/printbackends/libprintbackend-cups.a: Cannot stat: = No such file or directory tar: lib/gtk-2.0/2.10.0/printbackends/libprintbackend-cups.la: Cannot stat:= No such file or directory tar: lib/gtk-2.0/2.10.0/printbackends/libprintbackend-cups.so: Cannot stat:= No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 Creating package /pkg/All/gtk-2.18.7.tbz Registering depends: atk-1.28.0 cups-client-1.4.2_4 gnutls-2.8.3 libgcrypt-= 1.4.4 libgpg-error-1.7 pango-1.26.2 shared-mime-info-0.71 gio-fam-backend-2= =2E22.4 gamin-0.1.10_3 glib-2.22.4 gettext-0.17_1 libxml2-2.7.6_1 libiconv-= 1.13.1_1 libXinerama-1.0.3,1 libXi-1.2.1,1 libXrandr-1.3.0 libXcomposite-0.= 4.0,1 libXext-1.0.5,1 libXdamage-1.1.1 libXcursor-1.1.9_1 libXfixes-4.0.3_1= libXft-2.1.14 cairo-1.8.8,1 libXrender-0.9.4_1 libX11-1.2.1_1,1 xcb-util-0= =2E3.6_1 libxcb-1.5 libpthread-stubs-0.3_3 pcre-8.00 libXau-1.0.4 libXdmcp-= 1.0.2_1 xorg-fonts-truetype-7.4 font-bh-ttf-1.0.0 font-misc-meltho-1.0.0_1 = font-misc-ethiopic-1.0.0 mkfontdir-1.0.4 mkfontscale-1.0.6 libfontenc-1.0.4= xproto-7.0.15 bitstream-vera-1.10_4 fontconfig-2.8.0,1 freetype2-2.3.11 pi= xman-0.16.6 pkg-config-0.23_1 jasper-1.900.1_9 tiff-3.9.2_1 jbigkit-1.6 jpe= g-8 png-1.2.42 perl-5.10.1 python26-2.6.4 hicolor-icon-theme-0.12 expat-2.0= =2E1_1 encodings-1.0.2,1 font-util-1.0.1 compositeproto-0.4 damageproto-1.1= =2E0_2 fixesproto-4.0 inputproto-1.5.0 kbproto-1.0.3 randrproto-1.3.0 rende= rproto-0.9.3 xextproto-7.0.5 xineramaproto-1.1.2. Creating bzip'd tar ball in '/pkg/All/gtk-2.18.7.tbz' *** Error code 1 Stop in /ports/x11-toolkits/gtk20. *** Error code 1 Stop in /ports/x11-toolkits/gtk20. *** Error code 1 Stop in /ports/graphics/inkscape. fatal: failed Note that in most cases, the compilation *will* produce a package but will fail to produce packages for dependencies, resulting in pkg_add spewing errors upon install: pkg_add: could not find package liblqr-1-0.4.1 ! pkg_add: could not find package enchant-1.4.2 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! pkg_add: could not find package libxml++-2.26.1 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! pkg_add: could not find package libxml++-2.26.1 ! pkg_add: autoload of dependency '/pkg/All/cairomm-1.8.4.tbz' failed! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! pkg_add: could not find package libxml++-2.26.1 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! And so on and so on. What's the problem here? It surely shouldn't be this hard to use 'make pack= age'. The system is a vanilla install of FreeBSD 8.0 AMD64 with an up-to-date ports tree as of yesterday. xw From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 17:05:20 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B37C106566B for ; Fri, 26 Feb 2010 17:05:20 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id F363D8FC13 for ; Fri, 26 Feb 2010 17:05:19 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id B60B91FFC22; Fri, 26 Feb 2010 17:05:18 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id DA2BC845DB; Fri, 26 Feb 2010 18:05:17 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: xorquewasp@googlemail.com References: <20100226163227.GA15162@logik.internal.network> Date: Fri, 26 Feb 2010 18:05:17 +0100 In-Reply-To: <20100226163227.GA15162@logik.internal.network> (xorquewasp@googlemail.com's message of "Fri, 26 Feb 2010 16:32:27 +0000") Message-ID: <868wag53n6.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: package building failure irritation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 17:05:20 -0000 xorquewasp@googlemail.com writes: > tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or direc= tory > [...] > tar: Error exit delayed from previous errors. > pkg_create: make_dist: tar command failed with code 256 > Creating package /pkg/All/docbook-4.2.tbz > Registering depends: iso8879-1986_2 xmlcatmgr-2.2. > Creating bzip'd tar ball in '/pkg/All/docbook-4.2.tbz' > *** Error code 1 It looks like it's trying to create the package without first installing the port... which is weird, since "package" implies "install". You're sure it's not a permissions issue? Is your script running as root inside the jail? Is /usr/local writable by root inside the jail? You're not trying to build multiple ports in paralell? This would be easier to diagnose if you showed us the complete log for a single port... DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 17:19:56 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 381681065672 for ; Fri, 26 Feb 2010 17:19:56 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ew0-f226.google.com (mail-ew0-f226.google.com [209.85.219.226]) by mx1.freebsd.org (Postfix) with ESMTP id C11A18FC1B for ; Fri, 26 Feb 2010 17:19:55 +0000 (UTC) Received: by ewy26 with SMTP id 26so223615ewy.3 for ; Fri, 26 Feb 2010 09:19:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=G2PJ35xUq6N5GwTLsiQn+JJsYOQCHAy2AuT1V2Iudgw=; b=FcvcSzjsjYplpmlDdcxmmpnU0RvmC68CvvpSTwJVjbr+IuXcuZSoSvjxKcYS5cSCK8 dieyztVoReQ44fxnwJhYw0mXmOZAOPHlS/GxlMMVaZERuOYURvwdcuKYXSlRKP2gRJ45 4ltv/k6+Dbdwz+x1IXOZmEJffALukyGfN/5rA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=nC0i0JWhbFudhN4R3k806ZO/XesPWuZ+44gB7CWL8+0gev8cXjQuPi9WY2D0KZ2ILG k6ttWhKS5Xe3NVohDTYcdf7RpA98T1KvlZJS/uXoLT/aA5s8EUzBGagLfFEMw79Wnost QXLFBmlSNiiEeSIdfRHskKzpY/hl/8F8am9+k= MIME-Version: 1.0 Received: by 10.213.109.76 with SMTP id i12mr293708ebp.2.1267204791231; Fri, 26 Feb 2010 09:19:51 -0800 (PST) In-Reply-To: <7d6fde3d1002260300v36c420d4r79a18efe02c0d8ab@mail.gmail.com> References: <7d6fde3d1002252116k55a8e350s5dcc0846cba07785@mail.gmail.com> <7d6fde3d1002260300v36c420d4r79a18efe02c0d8ab@mail.gmail.com> Date: Fri, 26 Feb 2010 12:19:51 -0500 Message-ID: From: Ryan Stone To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD-Hackers Subject: Re: ddb and dump devices X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 17:19:56 -0000 It's a very simple custom protocol over UDP. You need a special server to listen for the netdump messages. Messing with something as complicated as NFS during a panic is going to increase the chances of a failure while trying to dump core. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 17:39:33 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE49A106564A for ; Fri, 26 Feb 2010 17:39:32 +0000 (UTC) (envelope-from glarkin@FreeBSD.org) Received: from mail1.sourcehosting.net (113901-app1.sourcehosting.net [72.32.213.11]) by mx1.freebsd.org (Postfix) with ESMTP id BB6CE8FC17 for ; Fri, 26 Feb 2010 17:39:32 +0000 (UTC) Received: from 68-189-245-235.dhcp.oxfr.ma.charter.com ([68.189.245.235] helo=cube.entropy.prv) by mail1.sourcehosting.net with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Nl49u-000EN9-AK; Fri, 26 Feb 2010 12:39:32 -0500 Received: from [127.0.0.1] (fireball.entropy.prv [192.168.1.12]) by cube.entropy.prv (Postfix) with ESMTP id F27AB3C13E2D; Fri, 26 Feb 2010 12:39:25 -0500 (EST) Message-ID: <4B88074E.7050007@FreeBSD.org> Date: Fri, 26 Feb 2010 12:39:26 -0500 From: Greg Larkin Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: xorquewasp@googlemail.com References: <20100226163227.GA15162@logik.internal.network> In-Reply-To: <20100226163227.GA15162@logik.internal.network> X-Enigmail-Version: 0.96.0 OpenPGP: id=1C940290 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Spam-Score: -0.3 (/) Cc: freebsd-hackers@freebsd.org Subject: Re: package building failure irritation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: glarkin@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 17:39:33 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 xorquewasp@googlemail.com wrote: > 'Lo. > > I've come up against an infuriating problem with ports. I don't even > know what causes it, let alone how to solve it. > > Essentially, to stop already-installed software from polluting port > builds, I build ports in a jail. I put a list of ports in a file and > use a trivial shell script to build them one after the other with: > > cd $port > make clean > make deinstall > make package-recursive > > The jail is set up with: > > /etc/make.conf: > WRKDIRPREFIX= /work > DISTDIR= /distfiles > PACKAGES= /pkg > PORTSDIR= /ports > > With /ports, /distfiles and /pkg being mounted from outside the > jail using nullfs (just because those directories exist on a ZFS > array outside of the jail for performance reasons). > > Before building a set of packages, the jail is wiped clean with > 'pkg_delete -a'. > > The problem is that it seems most ports just outright fail to > produce packages. I log each build and often the last lines of > the log files read something like: > > xchm: [...] > > Note that in most cases, the compilation *will* produce a package but will > fail to produce packages for dependencies, resulting in pkg_add spewing > errors upon install: > > pkg_add: could not find package liblqr-1-0.4.1 ! > pkg_add: could not find package enchant-1.4.2 ! > pkg_add: could not find package libsigc++-2.2.4.2 ! > pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! > pkg_add: could not find package libxml++-2.26.1 ! > pkg_add: could not find package libsigc++-2.2.4.2 ! > pkg_add: could not find package libsigc++-2.2.4.2 ! > pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! > pkg_add: could not find package libxml++-2.26.1 ! > pkg_add: autoload of dependency '/pkg/All/cairomm-1.8.4.tbz' failed! > pkg_add: could not find package libsigc++-2.2.4.2 ! > pkg_add: could not find package libsigc++-2.2.4.2 ! > pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! > pkg_add: could not find package libxml++-2.26.1 ! > pkg_add: could not find package libsigc++-2.2.4.2 ! > pkg_add: could not find package libsigc++-2.2.4.2 ! > pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! > > And so on and so on. > > What's the problem here? It surely shouldn't be this hard to use 'make package'. > > The system is a vanilla install of FreeBSD 8.0 AMD64 with an up-to-date > ports tree as of yesterday. > > xw Hi xw, "make package-recursive" calls "make package-noinstall" for all of the dependent ports. I'm assuming that means that the dependent packages all have to be installed first, so I would change your script like so: cd $port make clean make deinstall make depends # just to be safe make install make package-recursive On the other hand, this is similar to what Tinderbox (http://tinderbox.marcuscom.com/README.html), so perhaps that would be a solution for you, too? Hope that helps, Greg - -- Greg Larkin http://www.FreeBSD.org/ - The Power To Serve http://www.sourcehosting.net/ - Ready. Set. Code. http://twitter.com/sourcehosting/ - Follow me, follow you -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFLiAdO0sRouByUApARAhK0AKCjyH0T/n6lxt7w3fqaQM0dlh1CZQCgitmN 9R3WnYFpesYLW0mlZkVVzy0= =BCbS -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 18:07:49 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C072B1065676 for ; Fri, 26 Feb 2010 18:07:49 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from poseidon.ceid.upatras.gr (poseidon.ceid.upatras.gr [150.140.141.169]) by mx1.freebsd.org (Postfix) with ESMTP id 32B908FC15 for ; Fri, 26 Feb 2010 18:07:48 +0000 (UTC) Received: from mail.ceid.upatras.gr (unknown [10.1.0.143]) by poseidon.ceid.upatras.gr (Postfix) with ESMTP id DD5F8EB48CF; Fri, 26 Feb 2010 20:07:47 +0200 (EET) Received: from localhost (europa.ceid.upatras.gr [127.0.0.1]) by mail.ceid.upatras.gr (Postfix) with ESMTP id B9970160CE4; Fri, 26 Feb 2010 20:07:47 +0200 (EET) X-Virus-Scanned: amavisd-new at ceid.upatras.gr Received: from mail.ceid.upatras.gr ([127.0.0.1]) by localhost (europa.ceid.upatras.gr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TADJuhRCWyQt; Fri, 26 Feb 2010 20:07:47 +0200 (EET) Received: from kobe.laptop (ppp-94-64-198-255.home.otenet.gr [94.64.198.255]) by mail.ceid.upatras.gr (Postfix) with ESMTP id 7168E160CE8; Fri, 26 Feb 2010 20:07:47 +0200 (EET) Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.4/8.14.4) with ESMTP id o1QI7kpP003167 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Feb 2010 20:07:46 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Received: (from keramida@localhost) by kobe.laptop (8.14.4/8.14.4/Submit) id o1QI7iMg003146; Fri, 26 Feb 2010 20:07:44 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) From: Giorgos Keramidas To: Patrick Mahan In-Reply-To: <20100221202250.GF57731@acme.spoerlein.net> ("Ulrich \=\?iso-8859-1\?Q\?Sp\=F6rlein\=22's\?\= message of "Sun, 21 Feb 2010 21:22:51 +0100") Date: Fri, 26 Feb 2010 17:15:53 +0200 Message-ID: <873a0oqb86.fsf@kobe.laptop> References: <4B800F60.60700@mahan.org> <20100220210314.GB22800@kiwi.sharlinx.com> <20100221202250.GF57731@acme.spoerlein.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1.92 (berkeley-unix) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Cc: freebsd-hackers@freebsd.org Subject: Re: Building FreeBSD on a linux FC11 box. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 18:07:49 -0000 --=-=-= Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable On Sun, 21 Feb 2010 21:22:51 +0100, Ulrich Sp=F6rlein wro= te: > On Sat, 20.02.2010 at 13:03:14 -0800, R. Tyler Ballance wrote: >> You might want to ask the Debian GNU/kFreeBSD guys: >> http://www.debian.org/ports/kfreebsd-gnu/ >> >> I bet they've got a good idea :) > > They are using the kernel only, though. > > Before you reinvent the wheel, take a look at NetBSD, they are known for > their cross-compilation work. It's non-trivial to set up a NetBSD-like cross-compilation environment, but the bits you have to use are well-documented in the support of the NetBSD source tree for 'tools'. I've experimented a bit with some of the work that's needed and it includes at least the following: * mtree -- This is used by parts of the build glue to set up build-time directories. * make -- Targetting *all* possible platforms with BSD make is probably not worth the effort, but adding a minimal configure.in script that can produce an 'fbmake' binary suitable for cross building FreeBSD is doable and not very hard. With these two in place and a bit of Makefile target hacking it might be possible to cross-build FreeBSD on e.g. Linux. I don't have a Linux machine anymore, but if someone starts doing this I can probably set one up and test any patches. --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAkuH5bAACgkQ1g+UGjGGA7aB/wCggV7frZPeALfAwquxvvEUWDzc qrsAn0HtlnqZ/F6aztvmhPY/d0ClrB7G =Jfu8 -----END PGP SIGNATURE----- --=-=-=-- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 18:07:49 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE1911065675 for ; Fri, 26 Feb 2010 18:07:49 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from poseidon.ceid.upatras.gr (poseidon.ceid.upatras.gr [150.140.141.169]) by mx1.freebsd.org (Postfix) with ESMTP id 33AC18FC16 for ; Fri, 26 Feb 2010 18:07:48 +0000 (UTC) Received: from mail.ceid.upatras.gr (unknown [10.1.0.143]) by poseidon.ceid.upatras.gr (Postfix) with ESMTP id B9B10EB48C3; Fri, 26 Feb 2010 20:07:47 +0200 (EET) Received: from localhost (europa.ceid.upatras.gr [127.0.0.1]) by mail.ceid.upatras.gr (Postfix) with ESMTP id 9984B160CEB; Fri, 26 Feb 2010 20:07:47 +0200 (EET) X-Virus-Scanned: amavisd-new at ceid.upatras.gr Received: from mail.ceid.upatras.gr ([127.0.0.1]) by localhost (europa.ceid.upatras.gr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I97WTAvm17jB; Fri, 26 Feb 2010 20:07:47 +0200 (EET) Received: from kobe.laptop (ppp-94-64-198-255.home.otenet.gr [94.64.198.255]) by mail.ceid.upatras.gr (Postfix) with ESMTP id 33BA2160CE4; Fri, 26 Feb 2010 20:07:47 +0200 (EET) Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.4/8.14.4) with ESMTP id o1QI7jlJ003152 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Feb 2010 20:07:45 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Received: (from keramida@localhost) by kobe.laptop (8.14.4/8.14.4/Submit) id o1QI7imk003147; Fri, 26 Feb 2010 20:07:44 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) From: Giorgos Keramidas To: "Dr. Baud" In-Reply-To: <87341.15532.qm@web65601.mail.ac4.yahoo.com> (Baud's message of "Thu, 25 Feb 2010 11:09:36 -0800 (PST)") Date: Fri, 26 Feb 2010 17:28:27 +0200 Message-ID: <87y6igow2s.fsf@kobe.laptop> References: <87341.15532.qm@web65601.mail.ac4.yahoo.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1.92 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: debug libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 18:07:49 -0000 On Thu, 25 Feb 2010 11:09:36 -0800 (PST), "Dr. Baud" wro= te: > Apologies if this is the wrong list.... > > Are there prepackaged debug versions of the system libraries available > (like 'yum install *-debuginfo' in Fedora and 'apt-get install *-dbg' > in Ubuntu)? No, not really. You can always build a base system *with* debugging symbols from source though. Naturally, this is not as fast to install as a small set of pre-built packages, but it is at least possible. For example, my laptop runs a base system _with_ debugging symbols most of the time. To rebuild everything I made the following changes to my '/etc/make.conf' file: 1. I enabled the DEBUG_FLAGS option, by adding the following: DEBUG_FLAGS ?=3D -g 2. I checked that I am not using high optimization levels in CFLAGS or COPTFLAGS. 3. I enabled the NO_CPU_CFLAGS and NO_CPU_COPTFLAGS options: NO_CPU_CFLAGS=3D # Don't add -march=3D to CFLAGS auto= matically NO_CPU_COPTFLAGS=3D # Don't add -march=3D to COPTFLAGS a= utomatically Then I rebuilt everything from my latest source tree at /usr/src. The combined effect of NO_CPU_CFLAGS and DEBUG_FLAGS is that binaries are not stripped before they are installed, and optimization flags are not added automatically by the build system. Note #1. The resulting binaries are *much* larger. For example the ed(1) editor in unstripped format is 123 KB on my laptop, but stripping it reduces its size down to 48 KB: keramida@kobe:/tmp$ ls -ld /bin/ed -r-xr-xr-x 2 root wheel - 123663 16 =CE=A6=CE=B5=CE=B2 12:25 /bi= n/ed keramida@kobe:/tmp$ cp -a /bin/ed ed keramida@kobe:/tmp$ sudo strip -s ed keramida@kobe:/tmp$ ls -ld ed -r-xr-xr-x 1 keramida wheel - 48400 26 =CE=A6=CE=B5=CE=B2 17:24 ed The effect of enabling debugging flags for larger programs like gcc, gdb or bind is much much larger than ed(1) of course. Note #2. There's probably a measurable performance hit by running a debugging base system, because -O2 optimizations do not get a chance to do their magic as you build your binaries. HTH, Giorgos From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 18:15:21 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B13F71065673 for ; Fri, 26 Feb 2010 18:15:21 +0000 (UTC) (envelope-from xorquewasp@googlemail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 159868FC1A for ; Fri, 26 Feb 2010 18:15:20 +0000 (UTC) Received: by wyb40 with SMTP id 40so204808wyb.13 for ; Fri, 26 Feb 2010 10:15:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:received:received :x-authentication-warning:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to; bh=Qyou9qgfqpR/0m+DHjxVA+Wa62+r3ZVWmk2kzWEdOqY=; b=UMvt3vAHh9yCIJFI5eqEA/hekXTDloxfrG6EG1iG3n4qj9yB0+uVpjOO+wogeRvLFk RsPUMv/Ohj5F2ZdLXxuCvNPqfcNWTv+xI2vA17LG1drIR5g/J9FBv7tm5CQoT+x997SL cGc7m4caq4sgK50pXbNOEkcb2BHFao/7QVsrA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=x-authentication-warning:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to; b=fX5OIfUabEbXMA1Qvo7SXDc4459qqy18aSE96LaBYT0FV3Gs1Nq7rGt2CJnWC48460 nQ0eGZRuf2Q3ExhBLvUKviNo/jej0ql016c8YwFsm3hi3E2VuMOYvPdFCLknR2Dgynpl nRKG/NtoMOAV/T4DtZa6IuGXsa9BFZWI2A3YU= Received: by 10.216.89.149 with SMTP id c21mr668179wef.43.1267208114396; Fri, 26 Feb 2010 10:15:14 -0800 (PST) Received: from viper.internal.network ([78.143.196.85]) by mx.google.com with ESMTPS id n12sm1133985gve.15.2010.02.26.10.15.13 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 26 Feb 2010 10:15:13 -0800 (PST) Received: from viper.internal.network (localhost [127.0.0.1]) by viper.internal.network (Postfix) with ESMTP id 27A7F4AC06; Fri, 26 Feb 2010 18:15:12 +0000 (UTC) Received: (from m0@localhost) by viper.internal.network (8.14.3/8.14.3/Submit) id o1QIFBBx012048; Fri, 26 Feb 2010 18:15:11 GMT (envelope-from xorquewasp@googlemail.com) X-Authentication-Warning: viper.internal.network: m0 set sender to xorquewasp@googlemail.com using -f Date: Fri, 26 Feb 2010 18:15:11 +0000 From: xorquewasp@googlemail.com To: Greg Larkin Message-ID: <20100226181511.GC15162@logik.internal.network> References: <20100226163227.GA15162@logik.internal.network> <4B88074E.7050007@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B88074E.7050007@FreeBSD.org> Cc: freebsd-hackers@FreeBSD.org Subject: Re: package building failure irritation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 18:15:21 -0000 'Lo, On 2010-02-26 12:39:26, Greg Larkin wrote: > "make package-recursive" calls "make package-noinstall" for all of the > dependent ports. I'm assuming that means that the dependent packages > all have to be installed first, so I would change your script like so: > > cd $port > make clean > make deinstall > make depends # just to be safe > make install > make package-recursive I see! I'll edit the script and re-run the build to see if this sorts the problem. > On the other hand, this is similar to what Tinderbox > (http://tinderbox.marcuscom.com/README.html), so perhaps that would be a > solution for you, too? I used it up until about a week ago when I finally lost patience with it. Essentially, I found it too big, too slow, too opaque (too hard to work out what it was currently doing), required too much infrastructure (a working postgres setup - not good if you're trying to bootstrap packages for a new machine) and possibly most annoyingly, the distfile cache is still buggy (it redownloaded the 'qt-anywhere*' source archive about 20 times, making me exceed my monthly bandwidth limit). Oh, there were also serious problems building java packages in tinderbox (due to the 'agree to download' JDK garbage). I wish things were otherwise, but, oh well... xw From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 18:50:34 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5205C1065676; Fri, 26 Feb 2010 18:50:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0E4908FC16; Fri, 26 Feb 2010 18:50:34 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id AFAD746B6C; Fri, 26 Feb 2010 13:50:33 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 0CF188A025; Fri, 26 Feb 2010 13:50:33 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 26 Feb 2010 11:21:05 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <20100223215010.GA67619@triton8.kn-bremen.de> <20100225202850.GA79505@triton8.kn-bremen.de> In-Reply-To: <20100225202850.GA79505@triton8.kn-bremen.de> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002261121.05068.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 26 Feb 2010 13:50:33 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.4 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-emulation@freebsd.org, Tim Kientzle , Juergen Lock Subject: Re: 32 bit Linux lseek missing overflow check (was: Re: Linuxolator patches: stat and lseek SEEK_END for disk devices) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 18:50:34 -0000 On Thursday 25 February 2010 3:28:50 pm Juergen Lock wrote: > On Tue, Feb 23, 2010 at 10:50:10PM +0100, Juergen Lock wrote: > > Hi! > > > > Before this gets buried on -hackers in another thead... :) > > > > I now have disks appear as block devices for Linux processes (there > > already was commented out code for that in linux_stats.c, I hope my > > version is now `correct enough' to be usable [1]), and I made a simple > > patch to make lseek SEEK_END (L_XTND in the source) dtrt on disk > > devices too by simply invoking the DIOCGMEDIASIZE ioctl there; [2] > > both of these things are what (some) Linux processes expect. > > > > Patches are here: (made on stable/8, if they don't apply on head > > I'll have to make extra versions for that...) > > http://people.freebsd.org/~nox/linuxdisk-blk.patch [1] > > http://people.freebsd.org/~nox/lseek-seek_end.patch [2] > > > > And yes, with these patches the Linux bsdtar mentioned on -hackers > > in the `"tar tfv /dev/cd0" speedup patch' thread now also runs fast > > on FreeBSD. :) > > I now added an vn_isdisk() check to the second patch after comments from > julian, and I made a new patch that adds an overflow check to the 32 bit > linux lseek: (also at > http://people.freebsd.org/~nox/linux-lseek-overflow.patch > ) Hmm, when I asked Bruce, he actually said it was a feature that you didn't use vn_isdisk(). He also suggested that the proper way to fix lseek on devices is to fix stat in devfs to return a non-zero size. I have a possible fix to do that but haven't tested it yet: Index: devfs_vnops.c =================================================================== --- devfs_vnops.c (revision 204207) +++ devfs_vnops.c (working copy) @@ -44,6 +44,7 @@ #include #include #include +#include #include #include #include @@ -564,7 +565,11 @@ struct vattr *vap = ap->a_vap; int error = 0; struct devfs_dirent *de; + struct cdevsw *dsw; + struct thread *td; + struct file *fpop; struct cdev *dev; + off_t size; de = vp->v_data; KASSERT(de != NULL, ("Null dirent in devfs_getattr vp=%p", vp)); @@ -612,6 +617,18 @@ vap->va_ctime = dev->si_ctime; vap->va_rdev = cdev2priv(dev)->cdp_inode; + + dsw = dev_refthread(dev); + if (dsw != NULL) { + td = curthread; + fpop = td->td_fpop; + td->td_fpop = NULL; + if (dsw->d_ioctl(dev, DIOCGMEDIASIZE, (caddr_t)&size, + FREAD, td) == 0) + vap->va_size = size; + td->td_fpop = fpop; + dev_relthread(dev); + } } vap->va_gen = 0; vap->va_flags = 0; -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 18:50:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84F1D1065675 for ; Fri, 26 Feb 2010 18:50:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 569EF8FC2A for ; Fri, 26 Feb 2010 18:50:35 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 07B1D46B2D; Fri, 26 Feb 2010 13:50:35 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 2D6968A01F; Fri, 26 Feb 2010 13:50:34 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 26 Feb 2010 11:22:32 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <87341.15532.qm@web65601.mail.ac4.yahoo.com> In-Reply-To: <87341.15532.qm@web65601.mail.ac4.yahoo.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002261122.32597.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 26 Feb 2010 13:50:34 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.4 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: "Dr. Baud" Subject: Re: debug libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 18:50:35 -0000 On Thursday 25 February 2010 2:09:36 pm Dr. Baud wrote: > > Apologies if this is the wrong list.... > > Are there prepackaged debug versions of the system libraries available (like 'yum install *-debuginfo' in Fedora and 'apt-get install *-dbg' in Ubuntu)? No, however, you can build a debug version of any library by doing: cd src/lib/libfoo make clean make DEBUG_FLAGS=-g all install -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 21:11:57 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E988106566C; Fri, 26 Feb 2010 21:11:57 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id D601A8FC1C; Fri, 26 Feb 2010 21:11:56 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id E62D51E0014B; Fri, 26 Feb 2010 22:11:55 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1QL9qH6002505; Fri, 26 Feb 2010 22:09:52 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1QL9qvY002504; Fri, 26 Feb 2010 22:09:52 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Fri, 26 Feb 2010 22:09:52 +0100 To: John Baldwin Message-ID: <20100226210952.GA2253@triton8.kn-bremen.de> References: <20100223215010.GA67619@triton8.kn-bremen.de> <20100225202850.GA79505@triton8.kn-bremen.de> <201002261121.05068.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201002261121.05068.jhb@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Fri, 26 Feb 2010 21:19:22 +0000 Cc: freebsd-hackers@freebsd.org, freebsd-emulation@freebsd.org, Tim Kientzle , Juergen Lock Subject: Re: 32 bit Linux lseek missing overflow check (was: Re: Linuxolator patches: stat and lseek SEEK_END for disk devices) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 21:11:57 -0000 On Fri, Feb 26, 2010 at 11:21:05AM -0500, John Baldwin wrote: > On Thursday 25 February 2010 3:28:50 pm Juergen Lock wrote: > > On Tue, Feb 23, 2010 at 10:50:10PM +0100, Juergen Lock wrote: > > > Hi! > > > > > > Before this gets buried on -hackers in another thead... :) > > > > > > I now have disks appear as block devices for Linux processes (there > > > already was commented out code for that in linux_stats.c, I hope my > > > version is now `correct enough' to be usable [1]), and I made a simple > > > patch to make lseek SEEK_END (L_XTND in the source) dtrt on disk > > > devices too by simply invoking the DIOCGMEDIASIZE ioctl there; [2] > > > both of these things are what (some) Linux processes expect. > > > > > > Patches are here: (made on stable/8, if they don't apply on head > > > I'll have to make extra versions for that...) > > > http://people.freebsd.org/~nox/linuxdisk-blk.patch [1] > > > http://people.freebsd.org/~nox/lseek-seek_end.patch [2] > > > > > > And yes, with these patches the Linux bsdtar mentioned on -hackers > > > in the `"tar tfv /dev/cd0" speedup patch' thread now also runs fast > > > on FreeBSD. :) > > > > I now added an vn_isdisk() check to the second patch after comments from > > julian, and I made a new patch that adds an overflow check to the 32 bit > > linux lseek: (also at > > http://people.freebsd.org/~nox/linux-lseek-overflow.patch > > ) > > Hmm, when I asked Bruce, he actually said it was a feature that you didn't use > vn_isdisk(). He also suggested that the proper way to fix lseek on devices is > to fix stat in devfs to return a non-zero size. I have a possible fix to do > that but haven't tested it yet: > > Index: devfs_vnops.c > =================================================================== > --- devfs_vnops.c (revision 204207) > +++ devfs_vnops.c (working copy) > @@ -44,6 +44,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -564,7 +565,11 @@ > struct vattr *vap = ap->a_vap; > int error = 0; > struct devfs_dirent *de; > + struct cdevsw *dsw; > + struct thread *td; > + struct file *fpop; > struct cdev *dev; > + off_t size; > > de = vp->v_data; > KASSERT(de != NULL, ("Null dirent in devfs_getattr vp=%p", vp)); > @@ -612,6 +617,18 @@ > vap->va_ctime = dev->si_ctime; > > vap->va_rdev = cdev2priv(dev)->cdp_inode; > + > + dsw = dev_refthread(dev); > + if (dsw != NULL) { > + td = curthread; > + fpop = td->td_fpop; > + td->td_fpop = NULL; > + if (dsw->d_ioctl(dev, DIOCGMEDIASIZE, (caddr_t)&size, > + FREAD, td) == 0) > + vap->va_size = size; > + td->td_fpop = fpop; > + dev_relthread(dev); > + } > } > vap->va_gen = 0; > vap->va_flags = 0; > I had to add an D_DISK check, else it would panic at boot in a revoke syscall in ttydev_ioctl in ttydev_enter, apparently in the tty_lock call: (I didn't get a dump as it panic'd way too early in boot) http://fxr.watson.org/fxr/source/kern/tty.c#L159 Maybe dev->si_drv1 is null here? http://fxr.watson.org/fxr/source/kern/tty.c#L486 Anyway, with the D_DISK check added the patch seems to work: Index: src/sys/fs/devfs/devfs_vnops.c =================================================================== RCS file: /home/scvs/src/sys/fs/devfs/devfs_vnops.c,v retrieving revision 1.181.2.1 diff -u -p -p -u -r1.181.2.1 devfs_vnops.c --- src/sys/fs/devfs/devfs_vnops.c 3 Aug 2009 08:13:06 -0000 1.181.2.1 +++ src/sys/fs/devfs/devfs_vnops.c 26 Feb 2010 20:46:17 -0000 @@ -44,6 +44,7 @@ #include #include #include +#include #include #include #include @@ -563,7 +564,11 @@ devfs_getattr(struct vop_getattr_args *a struct vattr *vap = ap->a_vap; int error = 0; struct devfs_dirent *de; + struct cdevsw *dsw; + struct thread *td; + struct file *fpop; struct cdev *dev; + off_t size; de = vp->v_data; KASSERT(de != NULL, ("Null dirent in devfs_getattr vp=%p", vp)); @@ -611,6 +616,19 @@ devfs_getattr(struct vop_getattr_args *a vap->va_ctime = dev->si_ctime; vap->va_rdev = cdev2priv(dev)->cdp_inode; + + dsw = dev_refthread(dev); + if (dsw != NULL) { + td = curthread; + fpop = td->td_fpop; + td->td_fpop = NULL; + if ((dsw->d_flags & D_DISK) && + dsw->d_ioctl(dev, DIOCGMEDIASIZE, (caddr_t)&size, + FREAD, td) == 0) + vap->va_size = size; + td->td_fpop = fpop; + dev_relthread(dev); + } } vap->va_gen = 0; vap->va_flags = 0; From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 22:21:22 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FE821065673; Fri, 26 Feb 2010 22:21:22 +0000 (UTC) (envelope-from xorquewasp@googlemail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id ED67F8FC1B; Fri, 26 Feb 2010 22:21:21 +0000 (UTC) Received: by wyb40 with SMTP id 40so340398wyb.13 for ; Fri, 26 Feb 2010 14:21:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:received:received :x-authentication-warning:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to; bh=iFsos0B5kuuw0noT1koiCaocb/Nt9MRdPW7erbUuZK4=; b=SfKYGI6XE8j6LU/W6NKU7EPFV/FTGTEhQu9TYvjMBypCtFZs99uNzHdIv24hnLTT8P +/fRFMjKl1CUlhqMo7Vcv6jiXA9rgAAFcoRCLZ4GXp98b2BPTGXNGfXlOB4FIWPnZOuI BW1JKfR/D6dF9z8NvVvtRgGrPFZuHhlpbNHNM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=x-authentication-warning:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to; b=hNm5VNNIG3QbYEFHOf4CqWDrqctb1f0UjwI323RQjroKv78jT9s+XxeMV4B5t+CtOr zYfEDWWBp4SiuI3phfV1q2TQO1Y6HeCaKgjSOFo9kg0SEmY6iNW7JV0agGZ5CbqTdU6N tvbQ0ryTBdMipp85OmJS8KDG4Kq8U+oEdHmcQ= Received: by 10.216.163.67 with SMTP id z45mr871587wek.26.1267222875957; Fri, 26 Feb 2010 14:21:15 -0800 (PST) Received: from viper.internal.network (dsl78-143-196-85.in-addr.fast.co.uk [78.143.196.85]) by mx.google.com with ESMTPS id j8sm1887184gvb.1.2010.02.26.14.21.14 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 26 Feb 2010 14:21:15 -0800 (PST) Received: from viper.internal.network (localhost [127.0.0.1]) by viper.internal.network (Postfix) with ESMTP id 911094AC06; Fri, 26 Feb 2010 22:21:13 +0000 (UTC) Received: (from m0@localhost) by viper.internal.network (8.14.3/8.14.3/Submit) id o1QMLDeZ058133; Fri, 26 Feb 2010 22:21:13 GMT (envelope-from xorquewasp@googlemail.com) X-Authentication-Warning: viper.internal.network: m0 set sender to xorquewasp@googlemail.com using -f Date: Fri, 26 Feb 2010 22:21:13 +0000 From: xorquewasp@googlemail.com To: Greg Larkin Message-ID: <20100226222113.GA14592@logik.internal.network> References: <20100226163227.GA15162@logik.internal.network> <4B88074E.7050007@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B88074E.7050007@FreeBSD.org> Cc: freebsd-hackers@FreeBSD.org Subject: Re: package building failure irritation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 22:21:22 -0000 > cd $port > make clean > make deinstall > make depends # just to be safe > make install > make package-recursive This apparently didn't solve the problem (same errors). I'm stumped. xw From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 22:22:24 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 583CA106564A for ; Fri, 26 Feb 2010 22:22:24 +0000 (UTC) (envelope-from stephen@missouri.edu) Received: from cauchy.math.missouri.edu (cauchy.math.missouri.edu [128.206.184.213]) by mx1.freebsd.org (Postfix) with ESMTP id 1E1FD8FC1E for ; Fri, 26 Feb 2010 22:22:23 +0000 (UTC) Received: from laptop3.gateway.2wire.net (cauchy.math.missouri.edu [128.206.184.213]) by cauchy.math.missouri.edu (8.14.3/8.14.3) with ESMTP id o1QMMLr2003975; Fri, 26 Feb 2010 16:22:22 -0600 (CST) (envelope-from stephen@missouri.edu) Message-ID: <4B88499D.1010604@missouri.edu> Date: Fri, 26 Feb 2010 16:22:21 -0600 From: Stephen Montgomery-Smith User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100218 Firefox MIME-Version: 1.0 To: "xorquewasp@googlemail.com" References: <20100226163227.GA15162@logik.internal.network> <4B88074E.7050007@FreeBSD.org> <20100226181511.GC15162@logik.internal.network> In-Reply-To: <20100226181511.GC15162@logik.internal.network> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@FreeBSD.org" Subject: Re: package building failure irritation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 22:22:24 -0000 I also build my ports in a "jail" environment (only its not a really a jail, just a chroot). After I have built the ports I want, I create the packages using the following little script. (My mail client will have mangled the script, so take care when you copy and paste it, e.g. it needs to be tabs not spaces and the beginning of lines.) #!/usr/bin/make -f PACKAGES?=/usr/home/stephen/packages-8-amd PKG_DBDIR?=/var/db/pkg PORTSDIR?=/usr/ports PKG_LIST!=ls ${PKG_DBDIR} all: ${PKG_LIST:C+(.*)+${PACKAGES}/All/\1.tbz+} .for target in ${PKG_LIST} ${PACKAGES}/All/${target}.tbz: ${PKG_DBDIR}/${target}/+CONTENTS @origin=${PORTSDIR}/`sed -n 's/@comment ORIGIN://p' \ ${PKG_DBDIR}/${target}/+CONTENTS`; \ if [ $$origin != ${PORTSDIR}/ ] && [ -e $$origin ] && (! [ -e $$origin/work ] || [ -e $$origin/work/.install_done* ]); then \ echo creating package ${target}; \ make -C $$origin PACKAGES=${PACKAGES} package-links; \ pkg_create -GjY -b ${target} $@ || (rm -f $@ && false); \ fi .endfor .BEGIN: @mkdir -p ${PACKAGES}/All From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 22:38:03 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AF41106566B for ; Fri, 26 Feb 2010 22:38:03 +0000 (UTC) (envelope-from glarkin@FreeBSD.org) Received: from mail1.sourcehosting.net (113901-app1.sourcehosting.net [72.32.213.11]) by mx1.freebsd.org (Postfix) with ESMTP id 377FA8FC0A for ; Fri, 26 Feb 2010 22:38:02 +0000 (UTC) Received: from 68-189-245-235.dhcp.oxfr.ma.charter.com ([68.189.245.235] helo=cube.entropy.prv) by mail1.sourcehosting.net with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Nl8on-000NPq-9P; Fri, 26 Feb 2010 17:38:02 -0500 Received: from [127.0.0.1] (fireball.entropy.prv [192.168.1.12]) by cube.entropy.prv (Postfix) with ESMTP id 631473C16802; Fri, 26 Feb 2010 17:37:57 -0500 (EST) Message-ID: <4B884D48.90509@FreeBSD.org> Date: Fri, 26 Feb 2010 17:38:00 -0500 From: Greg Larkin Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: xorquewasp@googlemail.com References: <20100226163227.GA15162@logik.internal.network> <4B88074E.7050007@FreeBSD.org> <20100226222113.GA14592@logik.internal.network> In-Reply-To: <20100226222113.GA14592@logik.internal.network> X-Enigmail-Version: 0.96.0 OpenPGP: id=1C940290 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Spam-Score: -0.3 (/) Cc: freebsd-hackers@FreeBSD.org Subject: Re: package building failure irritation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: glarkin@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 22:38:03 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 xorquewasp@googlemail.com wrote: >> cd $port >> make clean >> make deinstall >> make depends # just to be safe >> make install >> make package-recursive > > This apparently didn't solve the problem (same errors). > > I'm stumped. > > xw Can you post a full log file of the process somewhere? The best way to capture it is with the commands: script /tmp/build.log # Starts a new shell # Run your script here and wait until it finishes exit # Saves the log file and # returns to the original shell dos2unix /tmp/build.log # Gets rid of ^M chars Then post /tmp/build.log somewhere for me to review. Thank you, Greg - -- Greg Larkin http://www.FreeBSD.org/ - The Power To Serve http://www.sourcehosting.net/ - Ready. Set. Code. http://twitter.com/sourcehosting/ - Follow me, follow you -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFLiE1H0sRouByUApARAv81AJ98rkAnQsfymc/HUpg5c6pt85eLsACfaZh2 SFcLzJAJPZBsIcWhyEqif0g= =USeg -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 26 22:50:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DC2A106564A for ; Fri, 26 Feb 2010 22:50:31 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (lefty.soaustin.net [66.135.55.46]) by mx1.freebsd.org (Postfix) with ESMTP id E637B8FC14 for ; Fri, 26 Feb 2010 22:50:30 +0000 (UTC) Received: by mail.soaustin.net (Postfix, from userid 502) id 782DB8C060; Fri, 26 Feb 2010 16:50:30 -0600 (CST) Date: Fri, 26 Feb 2010 16:50:30 -0600 From: Mark Linimon To: xorquewasp@googlemail.com Message-ID: <20100226225030.GA14194@lonesome.com> References: <20100226163227.GA15162@logik.internal.network> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100226163227.GA15162@logik.internal.network> User-Agent: Mutt/1.5.18 (2008-05-17) X-Mailman-Approved-At: Fri, 26 Feb 2010 23:00:10 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: package building failure irritation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 22:50:31 -0000 fwiw, the canonical way to find out if a port will package up in a clean environment (choort, with dependencies all loaded via package) is on http://pointyhat.freebsd.org/errorlogs/ , e.g., http://pointyhat.freebsd.org/errorlogs/i386-8-full/ . http://portsmon.freebsd.org gives you lots of cross-references into it and the PR database. Ignore the 'connection failed' message, it is transient and I need to fix it RSN. mcl From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 27 02:57:47 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7EAB1106564A for ; Sat, 27 Feb 2010 02:57:47 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-yx0-f177.google.com (mail-yx0-f177.google.com [209.85.210.177]) by mx1.freebsd.org (Postfix) with ESMTP id 2E54D8FC1F for ; Sat, 27 Feb 2010 02:57:46 +0000 (UTC) Received: by yxe7 with SMTP id 7so154106yxe.3 for ; Fri, 26 Feb 2010 18:57:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:in-reply-to:message-id:references:user-agent :x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; bh=9giYDuPlrvbYfYBgrgF8tzYclo39srF10vV/NAbmU5E=; b=LCdH/Qhqsifrs5yC+S9UdAYaZKosgJu+45YdPBOq/QoQMtFInhQEmatsrlUpyc/Dmo nFfJ9SPdaQ/G4Pkw3eX9Chi1P+sYZteAB26YT5xsthnxRGJfP6OeeLvFK625t0y+HuL3 TBh45tflwuZ4MtzBHMneCB1BDJilbTlHdcm8M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; b=WWlLxWYLowbWyggT3gz4PZK6F8PZ4pTeaWqg3qFYEqqUo1Q6DR7cHOvG+affwYrY+y mk7YqaJLrF/9QObZKZcRHMTIMpwcstjqCuTZsWPHryzVUOV6fdNEbSlxvGb/eQXuXTFQ ymhybbBQkGpWQfFlNqsDeECYMa9SVC0NvRHzk= Received: by 10.150.48.25 with SMTP id v25mr1113956ybv.240.1267239461242; Fri, 26 Feb 2010 18:57:41 -0800 (PST) Received: from ppp-21.16.dialinfree.com (ppp-21.16.dialinfree.com [209.172.21.16]) by mx.google.com with ESMTPS id 6sm272147yxg.30.2010.02.26.18.57.00 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 26 Feb 2010 18:57:39 -0800 (PST) Sender: "J. Hellenthal" Date: Fri, 26 Feb 2010 21:55:30 -0500 From: jhell To: Garrett Cooper In-Reply-To: <7d6fde3d1002260256g71b6e431i705cb8180fdd0a5d@mail.gmail.com> Message-ID: References: <7d6fde3d1002251850m3d32904emece0182e905b84c5@mail.gmail.com> <20100226092119.GA61498@owl.midgard.homeip.net> <7d6fde3d1002260203l5c7491c7w9ed84cdf40acf9d7@mail.gmail.com> <201002261229.10310.j.mckeown@ru.ac.za> <7d6fde3d1002260256g71b6e431i705cb8180fdd0a5d@mail.gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Hackers , Jonathan McKeown Subject: Re: mktemp(1) in /tmp or $PWD? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 02:57:47 -0000 On Fri, 26 Feb 2010 05:56, yanefbsd@ wrote: > On Fri, Feb 26, 2010 at 2:29 AM, Jonathan McKeown wrote: >> On Friday 26 February 2010 12:03:42 Garrett Cooper wrote: >>> FreeBSD is a great system; if there are ways that I can possibly make >>> it better by adding smart defaults >> >> Be careful about that value judgement. There are certainly ways you can change >> it. Not everyone might feel the change is for the better, and changing the >> default behaviour of widely-used commands is one change that you might find >> some people disagree with. > > I agree that there are some things that I'm going to have to apply on > my own personal machines to achieve behavior that I want, and I agree > that my way may not be the best way to do things, but that's part of > the reasoning why I'm asking whether or not others find some value in > what I'm proposing... > > Thanks, > -Garrett > Hi Garrett, First off don't get me wrong and I understand where your coming from on this but. GNU/Linux utilities have assumed the lazy system administrators approach for a long time, let me explain. By specifying defaults as said $PWD to find(1) you are now assuming that the user does not want to see any usage info and now has to type -h or maybe --help or possibly -? "This is Wrong". Now on the case for mktemp(1) always specifying a /tmp as a default is making the lazy assumption that /tmp will always be usable yes this is BSD, Linux, UNIX but any utility that is going to write and not have a specified path should always write to $CWD and if $CWD is not writable then its a programming mistake. I like knowing that when I issue a command whether it be from a script or from the prompt that I do not have to look elsewhere unless that is what I wanted. Regards, -- jhell From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 27 03:19:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF3361065670 for ; Sat, 27 Feb 2010 03:19:54 +0000 (UTC) (envelope-from a_best01@uni-muenster.de) Received: from zivm-exrelay3.uni-muenster.de (ZIVM-EXRELAY3.UNI-MUENSTER.DE [128.176.192.20]) by mx1.freebsd.org (Postfix) with ESMTP id 417928FC08 for ; Sat, 27 Feb 2010 03:19:53 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.49,549,1262559600"; d="txt'?scan'208";a="27058477" Received: from zivmaildisp1.uni-muenster.de (HELO ZIVMAILUSER04.UNI-MUENSTER.DE) ([128.176.188.85]) by zivm-relay3.uni-muenster.de with ESMTP; 27 Feb 2010 04:19:53 +0100 Received: by ZIVMAILUSER04.UNI-MUENSTER.DE (Postfix, from userid 149459) id 217551B07C1; Sat, 27 Feb 2010 04:19:52 +0100 (CET) Date: Sat, 27 Feb 2010 04:19:46 +0100 (CET) From: Alexander Best Sender: Organization: Westfaelische Wilhelms-Universitaet Muenster To: Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=+permail-2010022703194680e26a0b00002027-a_best01+ Cc: Subject: [patch] extending {amd64|i386} cpu info X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 03:19:54 -0000 This is a MIME encoded multipart message. --+permail-2010022703194680e26a0b00002027-a_best01+ Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit hi there, i always forget how to decipher model and family from the cpu id. of course there're 3rd party tools to do that for you, but instead i decided to hack the kernel. ;) maybe somebody finds these changes useful. cheers. alex --+permail-2010022703194680e26a0b00002027-a_best01+ Content-Type: text/plain Content-Transfer-Encoding: Base64 Content-Disposition: attachment; filename="identcpu.c.patch.txt" SW5kZXg6IHN5cy9hbWQ2NC9hbWQ2NC9pZGVudGNwdS5jCj09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9hbWQ2 NC9hbWQ2NC9pZGVudGNwdS5jCShyZXZpc2lvbiAyMDQzODMpCisrKyBzeXMvYW1kNjQvYW1kNjQv aWRlbnRjcHUuYwkod29ya2luZyBjb3B5KQpAQCAtMTg3LDcgKzE4NywxMiBAQAogCWlmIChjcHVf dmVuZG9yX2lkID09IENQVV9WRU5ET1JfSU5URUwgfHwKIAkgICAgY3B1X3ZlbmRvcl9pZCA9PSBD UFVfVkVORE9SX0FNRCB8fAogCSAgICBjcHVfdmVuZG9yX2lkID09IENQVV9WRU5ET1JfQ0VOVEFV UikgewotCQlwcmludGYoIiAgU3RlcHBpbmcgPSAldSIsIGNwdV9pZCAmIDB4Zik7CisJCXByaW50 ZigiICBTdGVwcGluZyA9ICV1IgorCQkgICAgICAgIiAgTW9kZWwgPSAldSIKKwkJICAgICAgICIg IEZhbWlseSA9ICV1IiwKKwkJICAgICAgIGNwdV9pZCAmIENQVUlEX1NURVBQSU5HLAorCQkgICAg ICAgQ1BVSURfVE9fTU9ERUwoY3B1X2lkKSwKKwkJICAgICAgIENQVUlEX1RPX0ZBTUlMWShjcHVf aWQpKTsKIAkJaWYgKGNwdV9oaWdoID4gMCkgewogCiAJCQkvKgpJbmRleDogc3lzL2kzODYvaTM4 Ni9pZGVudGNwdS5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9pMzg2L2kzODYvaWRlbnRjcHUuYwkocmV2 aXNpb24gMjA0MzgzKQorKysgc3lzL2kzODYvaTM4Ni9pZGVudGNwdS5jCSh3b3JraW5nIGNvcHkp CkBAIC02NzIsNyArNjcyLDEyIEBACiAJICAgIGNwdV92ZW5kb3JfaWQgPT0gQ1BVX1ZFTkRPUl9O U0MgfHwKIAkJKGNwdV92ZW5kb3JfaWQgPT0gQ1BVX1ZFTkRPUl9DWVJJWCAmJgogCQkgKChjcHVf aWQgJiAweGYwMCkgPiAweDUwMCkpKSB7Ci0JCXByaW50ZigiICBTdGVwcGluZyA9ICV1IiwgY3B1 X2lkICYgMHhmKTsKKwkJcHJpbnRmKCIgIFN0ZXBwaW5nID0gJXUiCisJCSAgICAgICAiICBNb2Rl bCA9ICV1IgorCQkgICAgICAgIiAgRmFtaWx5ID0gJXUiLAorCQkgICAgICAgY3B1X2lkICYgQ1BV SURfU1RFUFBJTkcsCisJCSAgICAgICBDUFVJRF9UT19NT0RFTChjcHVfaWQpLAorCQkgICAgICAg Q1BVSURfVE9fRkFNSUxZKGNwdV9pZCkpOwogCQlpZiAoY3B1X3ZlbmRvcl9pZCA9PSBDUFVfVkVO RE9SX0NZUklYKQogCQkJcHJpbnRmKCIgIERJUj0weCUwNHgiLCBjeXJpeF9kaWQpOwogCQlpZiAo Y3B1X2hpZ2ggPiAwKSB7Cg== --+permail-2010022703194680e26a0b00002027-a_best01+-- From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 27 09:34:20 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01721106568B; Sat, 27 Feb 2010 09:34:20 +0000 (UTC) (envelope-from xorquewasp@googlemail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5BDD68FC34; Sat, 27 Feb 2010 09:34:19 +0000 (UTC) Received: by wyb40 with SMTP id 40so529243wyb.13 for ; Sat, 27 Feb 2010 01:34:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:received:received :x-authentication-warning:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to; bh=q35EXpdT/lhc8zkpLSRCasAIZCE3uD4faCPyQ/0C+ss=; b=m2wmYDkYgR+XsawrHRZVGLoo2pwiD3VeZXnHN2Q4C+U3qjSt9yDpTGXS4lJEwxWpn9 C1kUoIVlgb3giZoIKdoxGlAcI9Hj7/XWPPR9AcXrkObi+JLJIaxQtTzRZ9NaLH3RhxYU cjC4TMOqZUY1O04JDABZsMiru9sWh8tIvcDsU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=x-authentication-warning:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to; b=K/aKvkvaFf2AEjfWMRjPd5eXPhKREoikaxx4RsQ1YdTbqTZU0vZdCzjcPjLdsICv9u Nv+QTQPg03t7/a/kXZKu7t3bYmuMIE0h8EnCR/iw3DgoxcOnnXz9OryIWiLf6cgesKmX V9qMDTVT2Orj+LXzjWacV16eZql2u7T6GF8oI= Received: by 10.216.88.83 with SMTP id z61mr1252403wee.14.1267263252952; Sat, 27 Feb 2010 01:34:12 -0800 (PST) Received: from viper.internal.network (dsl78-143-196-85.in-addr.fast.co.uk [78.143.196.85]) by mx.google.com with ESMTPS id g9sm3416338gvc.3.2010.02.27.01.34.11 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 27 Feb 2010 01:34:11 -0800 (PST) Received: from viper.internal.network (localhost [127.0.0.1]) by viper.internal.network (Postfix) with ESMTP id 216B24AC01; Sat, 27 Feb 2010 09:34:10 +0000 (UTC) Received: (from m0@localhost) by viper.internal.network (8.14.3/8.14.3/Submit) id o1R9Y9tp077463; Sat, 27 Feb 2010 09:34:09 GMT (envelope-from xorquewasp@googlemail.com) X-Authentication-Warning: viper.internal.network: m0 set sender to xorquewasp@googlemail.com using -f Date: Sat, 27 Feb 2010 09:34:09 +0000 From: xorquewasp@googlemail.com To: Greg Larkin Message-ID: <20100227093409.GA40858@logik.internal.network> References: <20100226163227.GA15162@logik.internal.network> <4B88074E.7050007@FreeBSD.org> <20100226222113.GA14592@logik.internal.network> <4B884D48.90509@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B884D48.90509@FreeBSD.org> Cc: freebsd-hackers@FreeBSD.org Subject: Re: package building failure irritation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 09:34:20 -0000 On 2010-02-26 17:38:00, Greg Larkin wrote: > Can you post a full log file of the process somewhere? The best way to > capture it is with the commands: 'Lo, I did actually send a log in response to DES' email but I sent the email from the wrong account and it still awaits list moderation. Here it is again: http://coreland.ath.cx/tmp/inkscape.txt xw From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 27 13:50:07 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 551E1106566B for ; Sat, 27 Feb 2010 13:50:07 +0000 (UTC) (envelope-from a_best01@uni-muenster.de) Received: from zivm-exrelay1.uni-muenster.de (ZIVM-EXRELAY1.UNI-MUENSTER.DE [128.176.192.14]) by mx1.freebsd.org (Postfix) with ESMTP id D22CA8FC17 for ; Sat, 27 Feb 2010 13:50:06 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.49,550,1262559600"; d="txt'?scan'208";a="297676735" Received: from zivmaildisp1.uni-muenster.de (HELO ZIVMAILUSER01.UNI-MUENSTER.DE) ([128.176.188.85]) by zivm-relay1.uni-muenster.de with ESMTP; 27 Feb 2010 14:50:05 +0100 Received: by ZIVMAILUSER01.UNI-MUENSTER.DE (Postfix, from userid 149459) id 18FC51B0768; Sat, 27 Feb 2010 14:50:05 +0100 (CET) Date: Sat, 27 Feb 2010 14:49:58 +0100 (CET) From: Alexander Best Sender: Organization: Westfaelische Wilhelms-Universitaet Muenster To: Message-ID: In-Reply-To: <20100222120300.22d4efb1@ernst.jennejohn.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=+permail-20100227134958f0889e840000246d-a_best01+ Cc: freebsd-hackers@FreeBSD.org Subject: Re: `make cleanworld` and /usr/obj/lib32/usr/src X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 13:50:07 -0000 This is a MIME encoded multipart message. --+permail-20100227134958f0889e840000246d-a_best01+ Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit how about something like this? alex Gary Jennejohn schrieb am 2010-02-22: > On Mon, 22 Feb 2010 00:20:14 +0100 (CET) > Alexander Best wrote: > > hi there, > > anybody fancy the idea of including /usr/obj/lib32/usr/src in the > > cleanworld > > target on amd64? > Seems like a reasonable thing to do. > --- > Gary Jennejohn --+permail-20100227134958f0889e840000246d-a_best01+ Content-Type: text/plain Content-Transfer-Encoding: Base64 Content-Disposition: attachment; filename="makefile.patch.txt" SW5kZXg6IE1ha2VmaWxlCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIE1ha2VmaWxlCShyZXZpc2lvbiAyMDQzODMp CisrKyBNYWtlZmlsZQkod29ya2luZyBjb3B5KQpAQCAtMTU4LDcgKzE1OCwxMiBAQAogCS1ybSAt cmYgJHsuT0JKRElSfS8qCiAJLWNoZmxhZ3MgLVIgMCAkey5PQkpESVJ9CiAJcm0gLXJmICR7Lk9C SkRJUn0vKgorLmlmIGV4aXN0cyAoJHtNQUtFT0JKRElSUFJFRklYfS9saWIzMiR7LkNVUkRJUn0v KQorCS1ybSAtcmYgJHtNQUtFT0JKRElSUFJFRklYfS9saWIzMiR7LkNVUkRJUn0vKgorCS1jaGZs YWdzIC1SIDAgJHtNQUtFT0JKRElSUFJFRklYfS9saWIzMiR7LkNVUkRJUn0KKwktcm0gLXJmICR7 TUFLRU9CSkRJUlBSRUZJWH0vbGliMzIkey5DVVJESVJ9LyoKIC5lbmRpZgorLmVuZGlmCiAKICMK ICMgSGFuZGxlIHRoZSB1c2VyLWRyaXZlbiB0YXJnZXRzLCB1c2luZyB0aGUgc291cmNlIHJlbGF0 aXZlIG1rIGZpbGVzLgo= --+permail-20100227134958f0889e840000246d-a_best01+-- From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 27 22:35:26 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3E73106564A for ; Sat, 27 Feb 2010 22:35:26 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from palm.hoeg.nl (mx0.hoeg.nl [IPv6:2001:7b8:613:100::211]) by mx1.freebsd.org (Postfix) with ESMTP id 67D258FC1C for ; Sat, 27 Feb 2010 22:35:26 +0000 (UTC) Received: by palm.hoeg.nl (Postfix, from userid 1000) id EEDF91CD23; Sat, 27 Feb 2010 23:35:24 +0100 (CET) Date: Sat, 27 Feb 2010 23:35:24 +0100 From: Ed Schouten To: Alexander Best Message-ID: <20100227223524.GU8200@hoeg.nl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OxWh+UR3R7TveXYN" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@FreeBSD.org Subject: Re: [patch] extending {amd64|i386} cpu info X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 22:35:26 -0000 --OxWh+UR3R7TveXYN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello Alexander, * Alexander Best wrote: > + printf(" Stepping =3D %u" > + " Model =3D %u" > + " Family =3D %u", > + cpu_id & CPUID_STEPPING, > + CPUID_TO_MODEL(cpu_id), > + CPUID_TO_FAMILY(cpu_id)); So is there some reason why we don't have a CPUID_TO_STEPPING()? Greetings, --=20 Ed Schouten WWW: http://80386.nl/ --OxWh+UR3R7TveXYN Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkuJniwACgkQ52SDGA2eCwWwFgCeJhHX0OH7OeJAbf6HzDLhH9Qn tTsAnjIo/zLqJK/1A21uQlMbbAawEYRn =oPmt -----END PGP SIGNATURE----- --OxWh+UR3R7TveXYN-- From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 27 23:01:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A4A1106566C for ; Sat, 27 Feb 2010 23:01:35 +0000 (UTC) (envelope-from a_best01@uni-muenster.de) Received: from zivm-exrelay2.uni-muenster.de (ZIVM-EXRELAY2.UNI-MUENSTER.DE [128.176.192.15]) by mx1.freebsd.org (Postfix) with ESMTP id A26B98FC19 for ; Sat, 27 Feb 2010 23:01:34 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.49,552,1262559600"; d="scan'208";a="237801981" Received: from zivmaildisp1.uni-muenster.de (HELO ZIVMAILUSER01.UNI-MUENSTER.DE) ([128.176.188.85]) by zivm-relay2.uni-muenster.de with ESMTP; 28 Feb 2010 00:01:32 +0100 Received: by ZIVMAILUSER01.UNI-MUENSTER.DE (Postfix, from userid 149459) id BAFF51B0768; Sun, 28 Feb 2010 00:01:32 +0100 (CET) Date: Sun, 28 Feb 2010 00:01:32 +0100 (CET) From: Alexander Best Sender: Organization: Westfaelische Wilhelms-Universitaet Muenster To: Ed Schouten Message-ID: In-Reply-To: <20100227223524.GU8200@hoeg.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org Subject: Re: [patch] extending {amd64|i386} cpu info X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 23:01:35 -0000 hmm...i guess the person who added the CPUID_TO_* macros to /usr/include/machine/specialreg.h thought a CPUID_TO_STEPPING macro wasn't necessary since getting the stepping from cpu_id can be done fairly easy without any shifting and such. a CPUID_TO_STEPPING macro would look something like this: #define CPUID_TO_STEPPING(id) \ ((id) & CPUID_STEPPING) getting the model and family however is more complicated since you have to combine CPUID_{MODEL|FAMILY} and CPUID_EXT_{MODEL|FAMILY}. cheers. alex Ed Schouten schrieb am 2010-02-27: > Hello Alexander, > * Alexander Best wrote: > > + printf(" Stepping = %u" > > + " Model = %u" > > + " Family = %u", > > + cpu_id & CPUID_STEPPING, > > + CPUID_TO_MODEL(cpu_id), > > + CPUID_TO_FAMILY(cpu_id)); > So is there some reason why we don't have a CPUID_TO_STEPPING()? > Greetings,