From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 13 21:17:19 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5B89D16A41F for ; Wed, 13 Jun 2007 21:17:19 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from mxout3.cac.washington.edu (mxout3.cac.washington.edu [140.142.32.166]) by mx1.freebsd.org (Postfix) with ESMTP id 3AB5513C465 for ; Wed, 13 Jun 2007 21:17:19 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from hymn01.u.washington.edu (hymn01.u.washington.edu [140.142.8.55]) by mxout3.cac.washington.edu (8.13.7+UW06.06/8.13.7+UW07.05) with ESMTP id l5DLHICu009213 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 13 Jun 2007 14:17:18 -0700 Received: from localhost (localhost [127.0.0.1]) by hymn01.u.washington.edu (8.13.7+UW06.06/8.13.7+UW07.03) with ESMTP id l5DLHIMa028677 for ; Wed, 13 Jun 2007 14:17:18 -0700 X-Auth-Received: from [192.55.52.1] by hymn01.u.washington.edu via HTTP; Wed, 13 Jun 2007 14:17:18 PDT Date: Wed, 13 Jun 2007 14:17:18 -0700 (PDT) From: youshi10@u.washington.edu To: hackers@freebsd.org In-Reply-To: <20070613182555.GA86571@keira.kiwi-computer.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-PMX-Version: 5.3.1.294258, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.6.13.135634 X-Uwash-Spam: Gauge=IIIIIII, Probability=7%, Report='SUPERLONG_LINE 0.05, NO_REAL_NAME 0, __CT 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0' Cc: Subject: Re: Using shell commands versus C equivalents X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2007 21:17:19 -0000 On Wed, 13 Jun 2007, Rick C. Petty wrote: > On Wed, Jun 13, 2007 at 10:23:36AM -0700, youshi10@u.washington.edu wrote: >> On Wed, 13 Jun 2007, Joerg Sonnenberger wrote: >> >> Should I briefly lock (flock) the file when running open/fstat/fchmod then >> to avoid issues? This may become a problem as pkg_*/make becomes more >> parallelized (another student's goals for his SoC project). > > I wouldn't bother. What issue are you trying to avoid? If one process is > trying to chmod +x and another is trying to do a chmod -x, it shouldn't > matter if you lock between the fstat/fchmod-- someone is going to win > anyway. This operation is not something that needs to be thread-safe. > >> Needless to say, pkg_* is by no means threadsafe in its current form >> though. It uses some global vars that are currently not mutex locked, and >> this type of file access is another issue (I wonder if spinlocking or >> sleeping waiting for flock to finish would be better in this case). > > Does pkg_* use multiple threads? I was under the impression each pkg tool > used a single thread (i.e. no threads) to do its operations and that they > wait for system(2)-type calls as needed. Maybe I'm not clear by what you > mean when you say "global vars". Well, I mean that as it currently stands there are several variables used globally for setting attributes per package (I'm not at my machine right now so I can't look them up until ~6pm PST). > Now another question is whether the pkg_* tools can handle multiple > processes managing the ports at the same time. For the mostpart, this is > true. Without looking at the code, I would expect that the only > contentions would be when trying to update the +REQUIRED_BY files. > Everything else should be just fine; you're not supposed to be installing > the same port multiple times at the exact same time, but maybe a lock could > be held on the package directory (i.e. /var/db/pkg/$PKG_NAME). Again, I > don't believe this is strictly necessary. Currently, no, and this is a condition that's contingent for a fellow SoC'er's project. The mentor said that all that *should* occur is there should be an flock, but that was it. So instead of making more work for him and since I am modifying pkg_* already, I thought it would be best to just make my modifications to simplify his end (he still has a ways to go on the dependency tracking I think). It goes a bit deeper than the +REQUIRED_BY files, in particular with the +CONTENTS, etc files as the pkg_* tools are enumerating the packages currently on the system, their dependencies, owning files, etc. Perhaps a global .lock file of some kind in the package directories would be the way to go though. Thanks, -Garrett