From owner-freebsd-arch@FreeBSD.ORG Sun Aug 21 02:24:58 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 64A0D106566B for ; Sun, 21 Aug 2011 02:24:58 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 449A68FC08 for ; Sun, 21 Aug 2011 02:24:58 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id p7L2OvTA093449 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 20 Aug 2011 19:24:57 -0700 (PDT) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id p7L2OvSH093448; Sat, 20 Aug 2011 19:24:57 -0700 (PDT) Received: from fbsd81 ([192.168.200.81]) by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA20962; Sat, 20 Aug 11 19:15:21 PDT Date: Sun, 21 Aug 2011 02:14:59 -0700 From: perryh@pluto.rain.com To: fullermd@over-yonder.net Message-Id: <4e50cc93.XBU8OPPxT6j9s9eK%perryh@pluto.rain.com> References: In-Reply-To: User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems/solutions: voting system & marketing surveys X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Aug 2011 02:24:58 -0000 > ... it's my understanding that there are significant legal > issues which prevent the Foundation from acting in this way > (taking targetted donations, acting as an escrow party on > specific bounties). If the problems are purely legal, there is likely a solution -- provided there is a willingness to look for it. Worst case, e.g. if the problem is a fundamental incompatibility with the Foundation's 501c3 status rather than merely a reluctance to undertake the necessary bookkeeping and other administrative tasks, it might be necessary to set up a separate legal entity to undertake those activities. FreeBSD would hardly be the first 501c3 entity to use that approach. From owner-freebsd-arch@FreeBSD.ORG Sun Aug 21 13:23:06 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26C1B106564A for ; Sun, 21 Aug 2011 13:23:06 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from fallbackmx07.syd.optusnet.com.au (fallbackmx07.syd.optusnet.com.au [211.29.132.9]) by mx1.freebsd.org (Postfix) with ESMTP id 015278FC13 for ; Sun, 21 Aug 2011 13:23:04 +0000 (UTC) Received: from mail27.syd.optusnet.com.au (mail27.syd.optusnet.com.au [211.29.133.168]) by fallbackmx07.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p7LB5SUc014461 for ; Sun, 21 Aug 2011 21:05:29 +1000 Received: from server.vk2pj.dyndns.org (c220-239-116-103.belrs4.nsw.optusnet.com.au [220.239.116.103]) by mail27.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p7LB5O7N006284 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 21 Aug 2011 21:05:26 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.4/8.14.4) with ESMTP id p7LB5Neq049648; Sun, 21 Aug 2011 21:05:23 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.4/8.14.4/Submit) id p7LB5MT8049647; Sun, 21 Aug 2011 21:05:22 +1000 (EST) (envelope-from peter) Date: Sun, 21 Aug 2011 21:05:21 +1000 From: Peter Jeremy To: Vadim Goncharov Message-ID: <20110821110521.GA48820@server.vk2pj.dyndns.org> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="huq684BweRXVnRxX" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Aug 2011 13:23:06 -0000 --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2011-Aug-17 23:10:19 +0000, Vadim Goncharov wro= te: >A month ago techlead of Rambler's Mail department declared in his blog=20 >that they begin migration from FreeBSD to Debian/Ubuntu. Comments to the= =20 >blog entry (there was much flame) revealed that search department of=20 >Russian search engine #1 Yandex.com also plans to migrate their=20 >search cluster - about 30,000 servers in several DCs (~60% of total=20 >their servers - to Linux. This is unfortunate. >The official reasons (really semi-official, as these are individual >blogs), in short, were: > * inadequate package manager and huge monolithic base system > * lack of OpenVZ-like virtualization (need CPU limiting) > * a FreeBSD marketshare forecast for 5 years I'll accept that the package management could be better and the base system is monolithic but it's hardly huge. It's not clear how marketshare forecast is relevant (though if they are as large as you imply, the marketshare forecast is circular). To take a devil's advocate approach, they need to accept some of the responsibility for what they perceive as FreeBSD shortcomings - they have chosen to not fund the development of functionality that they require and are now complaining that no-one else has either. >specialists on the market to hire (e.g. Yandex has about 20 FreeBSD >admins and about 84 Linux admins, for all other their services and >40% of servers). Based on this, a Linux system needs about 6.3 times as many admins as a FreeBSD system. It's not clear how migrating to Linux is going to save money here. And, again, they could always train people to meet their needs instead of expecting someone else to do it. > 1. Social (psychologic) problems of community (marketing, docs, ...). > >This is the most important one, because all technical problems are just >won't get solved because are even not viewed as problems. The FreeBSD >Project does not listen to users' needs. The typical response when poor >user want something is: "we don't need this, we won't change for you", >with "where are your patches?" at best. Then many users go out when see >such attitude toward them. I agree with this. The underlying problem is that most FreeBSD committers and developers are volunteers who work on FreeBSD in their spare time in areas that interest them. Even if they agree with the issue that the user raised, they probably don't have all of the time, expertise and motivation to work on it. Note that you can't tell a volunteer what to work on. If he doesn't want to work on something, he won't. If you push him, he'll just leave. If someone (person or corporation) wants some functionality that doesn't exist then they are going to need to either pay for it themselves or convince someone else (eg FreeBSD Foundation) to pay for it. Obviously, the FreeBSD Foundation is going to want to fund areas where it perceives it will get the greatest benefit. > 3. Ports and packages. > >What was the main problems with large-scale installations of FreeBSD in >that businesses? In short, that binary packages are not equal in rights >to ports, and that complicates things (i.e. requires too much work) when >one have many (> 10) servers. This was listed to me as: > >1) No pkg and pkg-devel versions. The -devel version is headers, static > libs, programmer examples, etc. not needed in production (we could > say this part is what is actually depended on in B-deps). Xorg is partially broken up in this way. In general, it is up to the ports' maintainers to do this - the FreeBSD project just hosts the ports infrastructure, it's up to maintainers to supply and maintain the actual ports. Note that requiring both pkg and pkg-devel versions of ports significantly increases maintainer effort for little (to them) perceived value. Also, I find having separate pkg and pkg-devel versions a real PITA - I regularly find that information i need is missing from the pkg file and I have to dig out the missing files. Out of interest, what is the rationale behind this requirement. =20 >2) Package name is dependent on options, so packages with another opts > don't work well when dependencies are rebuilt. This isn't always true and there are pros and cons to both approaches - making the package name independent of the options makes it very difficult to determine what options were used to build the package. This _is_ an area where the higher-level management tools (eg portmaster) can help. >3) Conflicts: no way to have apache13 and apache22 the same time. Because both install files with the same name. Again, this is an issue for the maintainer to address. >4) No dependence on base system. You may cut out something, recompile > world, deploy it on cluster and just then see that some packages are > now don't work. Ports _must_ depend on the base system or there's no point in having a base system. This is no different to changing any other port dependency and expecting it not to have any impact. I agree that there's no explicit tracking of base system dependencies and that is a deficiency. >5) Dependencies are badly designed. No version ranges in dependencies, > no alternative packages, no priorities in package search. I would dispute that dependencies are badly designed. There may be scope for allowing more flexibility in some cases but you then run the risk of running into subtle incompatibilities - especially since the maintainer can't be expected to verify all possible combinations. >6) Update problems. The version is just coded into name of package, and > dependencies are on the entire name, so there are situations when > install/upgrade of just one package may require rebuild 3/4 of all > pkgs. You cannot easiy modify installed package without editing pkgdb > manually. It is impossible to upgrade/replace package by out of the > box tools. Agreed. Tools like portmaster can assist here but it's not possible to always avoid rebuilding packages: If portx-3.1 depends on liby.so.2 in porty-2.4. When porty-2.4 gets upgraded so it installs liby.so.3, there is no alternative to rebuilding portx. It is unfortunate that a number of widely-used "core" ports have very unstable ABIs which are regularly changed but FreeBSD has no control over that. This is a no- win situation - if the ports are not updated, people complain that the ports are out of date. If the ports are updated, people complain about having to recompile all the affected ports. >7) Base system has no "out of the box" tools for package upgrade. Our > business opponents say this the least problem as one can always > install portupgrade, but conclude that overall base system concept > does not play well with full-featured packages (see also next part > about base system). And later on, you complain that the base system is too large. You can't have it both ways. Note that having the package upgrade system as a port means it can be be updated independently of the base system - which is a major benefit. >8) There is no -STABLE supported branches in ports. This issue comes up regularly. It's not clear how to achieve this without a major increase in infrastructure and committer resources. >It is obvious that current packages are not first-class citizens, in >comparison with ports. Source code is inherently more flexible than pre-compiled executables. There is no way to avoid this. >So packages need to be "equal in rights" in ports. The ports can have >things like this: > > .if ${OSVERSION} < 700104 || ${OSVERSION} >=3D 900000=20 Different versions of FreeBSD have different functionality. Sometimes this affects ports. > * Options must be included and installed to /var/db/ports/*/options > (this will allow to rebuild installed binary pkg as port) This is already done. > * Info about options must be included to /var/db/pkg/*/+CONTENTS like: > > @option WITH_SSL > @option WITHOUT_DEBUG > > * Dependencies must be able to specify needed OPTIONS, both required to > set and required to be unset, somewaht like: > > RUN_DEPENDS+=3D foobar:${PORTSDIR}/devel/foobar:+SSL:-SQLITE > > This will allow to detect conflicts with installed packages with > incompatible options. > > * For the package file names, introduce presets, e.g.: > > OPTIONS_PRESETS=3D default "+SSL:-DEBUG" \ > lite "-SSL:-DEBUG" These all seem like good ideas. > * (internal) move away from CVS, rebalance to category-subcategory. I don't believe the former point is relevant. There is probably scope for improving port categorisation. > 4. Base system, closely related with packages. =2E.. >2. Consequently, there is no way to check integrity (MD5 etc.) of any > non-RELEASE variant (freebsd-update IDS is very limited). If you have built a non-standard world, you need to generate/verify your own checksums. mtree(8) can do this and there are a number of other suitable tools in ports. >3. No ties between base system and packages: who knows what previous > admin has installed, you may have compiler or may not have, etc. > Packages may silently broke if some part of base system SUDDENLY > disappears, as no dependency information is recorded. This _is_ a deficiency. It's not clear how to cleanly resolve this. >4. Base system is monolithic, so there is no easy way to replace one > component with another - ports replacing base parts are hemorrhoids. OTOH, the base system is integrated together and works as a whole. This is a major advantage over Sinux. >Looking at /usr/src/sontrib and http://wiki.freebsd.org/ContribSoftware >we can identify many of what could became a package. There can be >different approaches to criteria "what is in base system": > >1. Only what is done on freebsd.org: all contrib must go to pkgs. I don't believe this is currently practical due to dependencies from core FreeBSD code on contrib code. >2. Whose effective vendor is now *@*bsd.org: contrib from other BSDs may > still live, and those with ceased upstream or renamed > (non-conflicting with ports) soft like libbsdxml, too. I'm not sure I follow this. >3. Axe out only the most odious contrib parts: BIND (as Peter said in > archives, host/dig could be resurrected from Attic), sendmail (could > be replaced by dma) and several others, presumably GCC/binutils & CVS > (I've also heard about painful Kereberos interferencing with Samba). This would seem to be a lose-lose situation. The workload on FreeBSD committers goes up as they take over all maintenance of local host/dig etc and users need to do more work to build a functional system. And you've already indicated you are opposed to having ports replace base components. >Axing out GCC to packages has another benefit: the newer GCC could be >used, and base could be polished to be more compiler-agnostic (hello, >clang). Work is currently under way to replace gcc with clang for building FreeBSD. I don't believe it's possible to build a truely compiler- agnostic system - FreeBSD is large and complex enough that it's highly likely to trigger compiler bugs. In order to deliver a reliable base OS, the compiler needs to be defined. >to split docs to packages, that's a right direction. For POLA reasons >all axed out packages (and sendmail too, respect traditions) should be >just packaged on CD1. Agreed. > 5. Too short major releases' cycle. =2E.. >It is known why the current scheme was adopted: the 4.11/5.3 case, >a horror. "Horror" is over-doing things. 3.x and 5.x both included major changes which made migration relatively difficult, as well as making it difficult to merge code back to the previous release. >Proposed solution: prolonging major releases fork time a little. Just to >time so only one stable branch will exist. I hope increasing branch >lifetime to one minor release will help: The difficulty of backporting fixes relates more to the magnitude of change in the code base than the time. It's not clear that increasing the longevity of major releases will really solve the problem - however long the Project supports branches, there will always be people complaining it's too short. > 6. Bug tracker, unicode and other less important trivia. > >GNATS is too old and unfriendly e.g. to user attachments. Agreed. But it's not clear what the solution is. >That's all for today. Thanks to everyone who has patience to carefully >read this all entirely. Thanks for your mail. It has made thought-provoking reading. --=20 Peter Jeremy --huq684BweRXVnRxX Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk5Q5nEACgkQ/opHv/APuIe0vgCcDoBI2d47ZkZBiU8dSrmrw+51 q60An0KY9ekLXzK36lB2nUUFbQ3Q9t/7 =ZVuU -----END PGP SIGNATURE----- --huq684BweRXVnRxX-- From owner-freebsd-arch@FreeBSD.ORG Sun Aug 21 13:52:01 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A0CE106564A for ; Sun, 21 Aug 2011 13:52:01 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 5B8B58FC13 for ; Sun, 21 Aug 2011 13:52:00 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Qv8Re-00050m-9s; Sun, 21 Aug 2011 17:52:10 +0400 Date: Sun, 21 Aug 2011 17:52:10 +0400 From: Slawa Olhovchenkov To: Peter Jeremy Message-ID: <20110821135210.GA62845@zxy.spb.ru> References: <20110821110521.GA48820@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110821110521.GA48820@server.vk2pj.dyndns.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: Vadim Goncharov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Aug 2011 13:52:01 -0000 On Sun, Aug 21, 2011 at 09:05:21PM +1000, Peter Jeremy wrote: > >specialists on the market to hire (e.g. Yandex has about 20 FreeBSD > >admins and about 84 Linux admins, for all other their services and > >40% of servers). > > Based on this, a Linux system needs about 6.3 times as many admins as > a FreeBSD system. It's not clear how migrating to Linux is going to > save money here. And, again, they could always train people to meet > their needs instead of expecting someone else to do it. Yandex try to reduce count of servers. Now [in Yandex] co-exist large set of productions servers and also large set of servers for tests and labs purpose. Migrating to Linux with virtualisation allow to drop tests and labs server and run on production cluster in off-peak time. > >5) Dependencies are badly designed. No version ranges in dependencies, > > no alternative packages, no priorities in package search. > > I would dispute that dependencies are badly designed. There may be > scope for allowing more flexibility in some cases but you then run > the risk of running into subtle incompatibilities - especially since > the maintainer can't be expected to verify all possible combinations. In real word this is [no backward compatibility] rare case. > >6) Update problems. The version is just coded into name of package, and > > dependencies are on the entire name, so there are situations when > > install/upgrade of just one package may require rebuild 3/4 of all > > pkgs. You cannot easiy modify installed package without editing pkgdb > > manually. It is impossible to upgrade/replace package by out of the > > box tools. > > Agreed. Tools like portmaster can assist here but it's not possible > to always avoid rebuilding packages: If portx-3.1 depends on liby.so.2 > in porty-2.4. When porty-2.4 gets upgraded so it installs liby.so.3, > there is no alternative to rebuilding portx. Also, this is rare case. More frequent liby.so.2 updated to liby.so.2.1, liby.so.2.2, liby.so.2.2.1 etc. > >7) Base system has no "out of the box" tools for package upgrade. Our > > business opponents say this the least problem as one can always > > install portupgrade, but conclude that overall base system concept > > does not play well with full-featured packages (see also next part > > about base system). > > And later on, you complain that the base system is too large. You > can't have it both ways. Note that having the package upgrade system > as a port means it can be be updated independently of the base system > - which is a major benefit. This is complain about missing pkg_update (since FreeBSD 3.x). Only pkg_add and pkg_delete, this is not preserve +REQUIRED_BY, for example. > > * Options must be included and installed to /var/db/ports/*/options > > (this will allow to rebuild installed binary pkg as port) > > This is already done. Realy?! I can install binary package by pkg_add and found build options in /var/db/ports/*/options? > >2. Consequently, there is no way to check integrity (MD5 etc.) of any > > non-RELEASE variant (freebsd-update IDS is very limited). > > If you have built a non-standard world, you need to generate/verify > your own checksums. mtree(8) can do this and there are a number of > other suitable tools in ports. Why is not done by installworld? > >3. No ties between base system and packages: who knows what previous > > admin has installed, you may have compiler or may not have, etc. > > Packages may silently broke if some part of base system SUDDENLY > > disappears, as no dependency information is recorded. > > This _is_ a deficiency. It's not clear how to cleanly resolve this. Define base as virtual packae with options (BASE_SSL, BASE_SENDMAIL etc) > >4. Base system is monolithic, so there is no easy way to replace one > > component with another - ports replacing base parts are hemorrhoids. > > OTOH, the base system is integrated together and works as a whole. > This is a major advantage over Sinux. Define 'negative' package as package remove files from base systems or other package. Allow package to replace files from base systems or other package. All this (removed or replaced files) must be preserved, From owner-freebsd-arch@FreeBSD.ORG Mon Aug 22 07:07:56 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60A52106566B for ; Mon, 22 Aug 2011 07:07:56 +0000 (UTC) (envelope-from damian@f21.pl) Received: from f21.pl (unknown [IPv6:2002:c21c:33bb::c21c:33bb]) by mx1.freebsd.org (Postfix) with SMTP id B167C8FC08 for ; Mon, 22 Aug 2011 07:07:55 +0000 (UTC) Received: from 194-28-51-187 ([194.28.51.187]) by f21.pl ; Mon, 22 Aug 2011 09:04:02 +0200 Message-ID: From: "Damian Puchalski" To: arch@freebsd.org Date: 22 Aug 2011 09:04:02 +0200 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Cooperation X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2011 07:07:56 -0000 =0D=0A =0D=0A Hi, = =0D=0A My name is Damian Puchalski, is the owner of a software house in= Poland. =0D=0A Generally we take care of developing web applications= in Microsoft .NET =0D=0A language. =0D=0A =0D=0A But, I am not writing= for this. Seeing that Poland is now experiencing a =0D=0A reality of great= economic growth, many products served in shops, mall, =0D=0A internet does= not satisfy all people. =0D=0A I would suggest that you can invest,= and enter your products / services in =0D=0A our market. =0D=0A = =0D=0A From my side I can give you: =0D=0A - Consultation (management,= organization) =0D=0A - Human resources =0D=0A - Office / shop = =0D=0A - Server data center in Poland =0D=0A - Propagation of products= / services =0D=0A =0D=0A To give an example of prices. Production= banking software company, the =0D=0A initial phase of propagation of they= services in Poland has been subjected =0D=0A to the following monthly costs: = =0D=0A - Office 70m2 - 500 euro =0D=0A - Three JAVA programmers - 5000= euro =0D=0A - A Team Leader - 2000 euro =0D=0A - Internet connection,= electricity, other - 60 euro =0D=0A - Marketing - 1500 euro =0D=0A= =0D=0A Total: 9060 euro =0D=0A Our commission: 15% - 1359 euro = =0D=0A Total II: 10419 euro =0D=0A =0D=0A The rates in Poland: = =0D=0A VAT - 23% =0D=0A Company Tax - 18% =0D=0A =0D=0A A simulation= of gross earnings / net: =0D=0A Earnings gross: 100000 euro (123000= + VAT) =0D=0A Costs: 80000 euro (98400 euro + VAT) =0D=0A Net earnings:= 20000 euro =0D=0A Fees: 3600 euro =0D=0A Net earnings II: 16400 euro = =0D=0A VAT Return: 0 Euro (VAT is returned if Costs > Earnings) =0D=0A= =0D=0A Total net income: 16400 euro =0D=0A =0D=0A =0D=0A= =0D=0A If your company has the need, I can be helpful. =0D=0A = =0D=0A =0D=0A = =0D=0A =0D=0A ---------------------------------------------------------- Software= =0D=0AHouse //F21 .NET R&D Poland Damian Puchalski Owner Skype:= =0D=0Adamian.torino E-mail: [1]damian@f21.pl I nostri punti forti che ci differiscono dalla concorrenza: -= Usiamo =0D=0A Technologia avanzata Microsoft - Prendiamo cura e responsabilita= =0D=0A economica da scelte e progetti che sviluppiamo - Nostri servizzi= sono =0D=0A indirizziato per tutte le societa d' europa - Impiegiamo= solamente =0D=0A migliori risorse e collaboriamo solamente con migliori = =0D=0A References 1. 3D"mailto:damian@f21.pl" From owner-freebsd-arch@FreeBSD.ORG Mon Aug 22 17:05:08 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FA1D106566C for ; Mon, 22 Aug 2011 17:05:08 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0D59B8FC0A for ; Mon, 22 Aug 2011 17:05:07 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QvXvt-00056a-11 for freebsd-arch@freebsd.org; Mon, 22 Aug 2011 19:05:05 +0200 Received: from static-78-8-147-77.ssp.dialog.net.pl ([78.8.147.77]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 22 Aug 2011 19:05:05 +0200 Received: from mwisnicki+freebsd by static-78-8-147-77.ssp.dialog.net.pl with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 22 Aug 2011 19:05:05 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Marcin Wisnicki Date: Mon, 22 Aug 2011 17:04:54 +0000 (UTC) Lines: 18 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: static-78-8-147-77.ssp.dialog.net.pl User-Agent: Pan/0.134 (Wait for Me; Unknown) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2011 17:05:08 -0000 On Thu, 18 Aug 2011 07:49:30 +0000, Baptiste Daroussin wrote: >> >> * Options must be included and installed to /var/db/ports/*/options >> (this will allow to rebuild installed binary pkg as port) >> >> > This could be a good idea :) I'll keep it for pkgng > See also +BUILD_INFO and other metadata files in pkgsrc. I couldn't find documentation but you can grab random binary package and inspect its contents. It's a shame that pkgsrc is so often ignored here. It solves so many problems of ports and is so far ahead that any effort to improve ports will never match what is already available from pkgsrc. From owner-freebsd-arch@FreeBSD.ORG Mon Aug 22 18:34:00 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A56DC106564A for ; Mon, 22 Aug 2011 18:34:00 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 36CA38FC0A for ; Mon, 22 Aug 2011 18:34:00 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QvZJu-0000C5-98 for freebsd-arch@freebsd.org; Mon, 22 Aug 2011 20:33:58 +0200 Received: from static-78-8-147-77.ssp.dialog.net.pl ([78.8.147.77]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 22 Aug 2011 20:33:58 +0200 Received: from mwisnicki+freebsd by static-78-8-147-77.ssp.dialog.net.pl with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 22 Aug 2011 20:33:58 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Marcin Wisnicki Date: Mon, 22 Aug 2011 18:33:45 +0000 (UTC) Lines: 37 Message-ID: References: <20110821110521.GA48820@server.vk2pj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: static-78-8-147-77.ssp.dialog.net.pl User-Agent: Pan/0.134 (Wait for Me; Unknown) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2011 18:34:00 -0000 On Sun, 21 Aug 2011 21:05:21 +1000, Peter Jeremy wrote: > On 2011-Aug-17 23:10:19 +0000, Vadim Goncharov > wrote: >>1) No pkg and pkg-devel versions. The -devel version is headers, static >> libs, programmer examples, etc. not needed in production (we could >> say this part is what is actually depended on in B-deps). > > Xorg is partially broken up in this way. In general, it is up to the > ports' maintainers to do this - the FreeBSD project just hosts the ports > infrastructure, it's up to maintainers to supply and maintain the actual > ports. Note that requiring both pkg and pkg-devel versions of ports > significantly increases maintainer effort for little (to them) perceived > value. Also, I find having separate pkg and pkg-devel versions a real > PITA - I regularly find that information i need is missing from the pkg > file and I have to dig out the missing files. > > Out of interest, what is the rationale behind this requirement. > I too find lack of -devel packages as one of freebsd strengths not weaknesses. Such separation is also very specific to certain languages like C/C++. However to provide a middle-ground solution I once proposed installation filters based on patterns, which would give ability to not have unwanted files essentially for free (just small changes in pkg_* and ports/Mk). For example there could be a standard filter group called "devel" that includes "include/**" and "lib/**.a". Packages would have ability to exclude/include additional files to any group if needed using pkg-plist directives. Similar patterns could be defined for docs, localizations, etc. User would set which groups of files he wants to exclude during installation or after it. Of course ideas don't write code :( From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 06:37:27 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 857C11065670 for ; Tue, 23 Aug 2011 06:37:27 +0000 (UTC) (envelope-from pcthegreat@gmail.com) Received: from mail-pz0-f45.google.com (mail-pz0-f45.google.com [209.85.210.45]) by mx1.freebsd.org (Postfix) with ESMTP id 5A0B88FC08 for ; Tue, 23 Aug 2011 06:37:27 +0000 (UTC) Received: by mail-pz0-f45.google.com with SMTP id 33so18952136pzk.18 for ; Mon, 22 Aug 2011 23:37:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LvIuvbB6QuZu+Bs45U+BAt9R4cF6HWHL+om4ufcUwHw=; b=vwOrpBiWHTun8cMIm63rW2a2AHyHbijY0gQNC9EzYEeCCLnKroB/NErKIQYaSjv/+f Jx4TZetl66LgDzORS9UDA4NA3JCrrt73gDBxINmge6wZn2yHH4fq2P8Pnyl3/Qg/Oh6Y yzPvODE/72r2LbmiSrohjZEq98MC1oqB6zX28= MIME-Version: 1.0 Received: by 10.142.225.4 with SMTP id x4mr1715836wfg.86.1314079584873; Mon, 22 Aug 2011 23:06:24 -0700 (PDT) Received: by 10.68.47.101 with HTTP; Mon, 22 Aug 2011 23:06:24 -0700 (PDT) In-Reply-To: References: Date: Tue, 23 Aug 2011 10:06:24 +0400 Message-ID: From: selven To: vadim_nuclight@mail.ru Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 06:37:27 -0000 Good post as well as pretty much scary. You mentionned about the purity of FreeBSD at some point, that's one of the reasons why i prefer FreeBSD (probably many of my friends who do work with it also), lately, with all those ubuntu fans, trying to get the company to keep using FreeBSD has become pretty much a hassle. Since in a technical board, with 10 people, 7 out of 10 are ubuntu users, with votes one always loses, and the funny thing :s 7 out of 10 don't even know the difference between FreeBSD and linux, they just voice out ubuntu because 'you will end up in trouble finding people to maintain your stuffs if these 3 leaves' (weirdly we even fixes there buntu bugs :s ). This user base is really a huge problem. On Thu, Aug 18, 2011 at 3:10 AM, Vadim Goncharov wrote: > Hi, > > I have bad news. > > A month ago techlead of Rambler's Mail department declared in his blog > that they begin migration from FreeBSD to Debian/Ubuntu. Comments to the > blog entry (there was much flame) revealed that search department of > Russian search engine #1 Yandex.com also plans to migrate their > search cluster - about 30,000 servers in several DCs (~60% of total > their servers - to Linux. The problem here is that both big companies > were using FreeBSD from the beginning (~1997), so migration will be > rather expensive. > > The official reasons (really semi-official, as these are individual > blogs), in short, were: > * inadequate package manager and huge monolithic base system > * lack of OpenVZ-like virtualization (need CPU limiting) > * a FreeBSD marketshare forecast for 5 years > > Despite of several our committers working for them, the operating > expenses for FreeBSD are considered too high by these companies. They've > not confirmed, but the key problem is probably shortage of FreeBSD > specialists on the market to hire (e.g. Yandex has about 20 FreeBSD > admins and about 84 Linux admins, for all other their services and > 40% of servers). So this is problem of FreeBSD's too low > userbase/marketshare, caused, in turn, by other FreeBSD's problems. > > Given that they is just planning today, but not yet started, we have to > solve our problems or FreeBSD will effectively die[1]. Currently > https://ssl.netcraft.com/ssl-sample-report//CMatch/oscnt_all shows that > still 3% of servers are running FreeBSD. We are currently probably at > the tip of 'hype curve'[2] and then our userbase will begin to shrink, > and the task is to solve problems, so after hard time it will rise again > (hopefully more than it was before). > > [1] 'Die' means shrinking userbase size to those of NetBSD/OpenBSD. > [2] http://www.metodolog.ru/01493/2.gif > > Earlier this year in this list marcel@ have said about objective view: > http://lists.freebsd.org/pipermail/freebsd-arch/2011-January/010999.html > from the outside, not FreeBSD people. I have posted a call to Russian > community in my blog (http://nuclight.livejournal.com/128712.html) and > gathered a list of problems along with possible solutions to some of > them. The rest of this message is summary of them. > > In general, people sympathizing FreeBSD agree that there are no fatal > technical problems and that FreeBSD could compete with mainstream Linux. > That said, the 80/20 principle is in effect and 80% losed users for > FreeBSD are caused by 20% problems. The trouble here is that what > "outside" people view as considerable problems, we here inside FreeBSD > view as something insignificant ("do it yourself"). Thus 80% could be > achieved by 20% of work. I have sorted them in order of decreased > importance (priority): > > 1. Social problems of community (and marketing, docs, ...) > 2. Lack of drivers and virtualization. > 3. Ports and packages. > 4. Base system, closely related with packages. > 5. Too short major releases' cycle. > 6. Bug tracker, unicode and other less important trivia. > (bonus) FreeBSD strengths to be concentrated on. > > Now all of them more detailed. > > > 1. Social (psychologic) problems of community (marketing, docs, ...). > > This is the most important one, because all technical problems are just > won't get solved because are even not viewed as problems. The FreeBSD > Project does not listen to users' needs. The typical response when poor > user want something is: "we don't need this, we won't change for you", > with "where are your patches?" at best. Then many users go out when see > such attitude toward them. > > The key points are: > > 1) *The competent user is not zealot*. > 2) The system is *for users, not for developers*. > > With first: when user sees the system costs too much time for him, and > that those problems won't get fixed and even considered such - he will > switch to another OS. This may mean that he is able to follow our advice > hoe to do some or another thing (e.g. to recompile all ports), but this > is unacceptable to him because this is too much maintenance (another > system by objectivity requires less work). Many businesses fall into > this category. They will not with FreeBSD by any price just because they > love OS. Unlike ourselves. > > With second: that's simple and not simple. It is simple in that by > nature each person don't make a work just for himself but for all > others, who are now "users" to him, too - just by induction that's not > for developers only. It is not simple due to question "Why do we need > those who don't contribute anything back to us?" which random committer > has right to ask. The answer, in short, is: there tasks which could be > only done by large groups of people, e.g. big corporations. For > corporations to invest and contribute back - the system must be popular > enough. To be popular enough means there must be a big amount of users > who is not contributing anything. It is useful in itself, though, that > some of those users, say 1%, will become contributors, that is, absolute > number of our developers will also benefit from FreeBSD being much more > popular. > > There are opinions in our community like: > > | > Personally I'd like to think that that we write an OS for users. > | > | We write an OS for the people who can and will use an OS written by > | us. > > | FreeBSD is whatever its developers make it be > > These are wrong and harmful nowadays. The world has changed. We have to > admit it or we will die. We have to admit mistakes and *change* some of > the ways we're doing things: any answer like "I don't agree, we don't > need these users/features/etc., we *won't* do this for someone" - is > just another step to grave. > > Of course that doesn't mean to do everything - we often have not enough > time/resources to do. But that's another question and should not be > mixed with attitude - "it's good but we have no resources, will you do?" > vs "we don't need this, go away" (that's not only about features but > about keeping something too). > > So ("system is for users") we need to (re)define who our user is. I view > the current situation as following: > > 100% .---------------. The stratification of overall > | Ubuntu | our userbase. Areas are: > | target | 6 > | audience | 1 - official developers > expand -> |---------------| > to here | not our, but | 2 - actively participating and > | competent | 5 sending patches, contributors > | users | and other "zealots" (whose > |===============| "soul is with FreeBSD") > |announce@ only | 4 > involved |---------------| 3 - not so active but discussing > | subscribed to | in mail lists, patches go > | our mailing | 3 sometimes > | lists | > committed |- - - - - - - -| 4 - busy with work to read lists > | | 2 > |_______________| 5 - can use, but costs are now high > | *@freebsd.org | 1 > 0% `---------------' 6 - we'll never want them > > The terms "involved" and "committed" were taken from > http://en.wikipedia.org/wiki/The_Chicken_and_the_Pig for those who are > "at the heart" of the community, at "periphery" and all others. > Currently, only those in (1) and partially (2) are listened to, > sometimes with "call for ..." in (3). Meanwhile, the most important of > our users, who are using FreeBSD in production, who are busy with real > work in real world, are in (4) and sometimes (5). Those are largely > ignored by the Project - even messages to announce@ last years are less > about important events and calls for volunteers/money. > > The other social problem is lack of companies which offer commercial > support of FreeBSD like RedHat does. > > As a possible solution to find out what user needs are, and to > compensate the fact that not all users in (4) and (5) care enough about > our principles and spirit, I propose a system for "weighted democracy". > This is a voting system (I could implement it), say, there is a mail > voting@freebsd.org to which users send filled in vote forms, with > selected answers from a survey published in announce@. > > The system has a database where users are recorded given their FreeBSD > activity in mail lists etc., and votes are summed as follows: > > + 1.00 vote for anyone not in database (not involved to FreeBSD) > + a decimal logarithm of number of posts to mail lists (2 votes for 100 > posts, 3 votes for 1000 posts) > + a binary logarithm of num PRs in GNATS db (6 votes for 64 send-pr's) > + a proportion (say, N/2) of entries for this person in "svn log" (e.g. > in "Submitted by:") > + an assigned (by core@) number of votes in special exceptional DB (for > corporation and the like) > > The system then presents results for each answer: 1) how many users > voted, 2) how many votes summed. > > This is by no mean to measure "exact contribution", but to defend from > anonymous users and trolls who not care about amount of work developers > will need to. The results are viewed as a feedback to core team, not as > a final decision - that's all for what marketing is solely exist. There > no adequate feedback from users to developers currently at all - > individual posts in mail lists are too small for statistics (what about > ministat(1) for users, huh?..). > > What is also in this part? Documentation. No, not in that sense. The > observations show that while there are described solutions to problems > in system, it's difficult for users to find it. It's somehow organized > or misorganized that solution is not intuitive for them (e.g. someone > migrating from Windows complained that it was difficult to find 'simply > how to format a flash drive' in Handbook). I don't know how to handle > this properly. May be catalogues with Best Practices howto's pointing to > exact sections of Handbook, FAQ, articles etc.? May be better search > mechanisms?.. > > And the last here about too much conservatism and rigidness in our camp. > As one of the opponents said (roughly translated): > > | FreeBSD, historically, was good system, let's say, "valid system > | for solid people". Pureness, strictness, etc. The base system is > | consequence of this approach. The trouble is, that in 2011 nobody > | needs pureness and strictness. Customers need transparency and fast > | reaction to their requests. If there is no transparency and speed, > | you get CentOS 6 (just released, at last). Or FreeBSD (Gentoo, > | Solaris, you name it). > > I don't agree that those certainly contradicts each other, but we > definitely need to change, er, something. And don't keep something just > because it is tradition, if that tradition has no more technical reasons > for our users. Let's shift conservatism to another field where it is > really needed (e.g. legacy support and other examples below). > > Finally, two more quotes from the arch@freebsd.org archives of last > 2 years: > > | From: Cyrille Lefevre > | [...] however, you answer remember me why I quit FreeBSD, cease > | fire, courtesy is the key > > and > > | From: Julian H. Stacey > | Though I & many others have sent lots of send-pr, contributing even > | a spelling correction to FreeBSD is much harder than to e.g. > | http://wikipedia.org > > The threshold of entering to FreeBSD is too high and should be lowered. > I can confirm the last quote on my own example, 2 months ago. I've > submitted a patch to ipfw, adding call/return rule actions. It allows > to organize rules in somewhat like pf anchors, and, more importantly, > iptables chains, enabling FreeBSD ipfw to compete with Linux at least > partially in this sphere. The patch wasn't taken in maillists, I had > to push it in IRC, and push hard: the whole needed by many big users > thing was deferred (and still not MFCed to 8.x) just because some > #define's and printf's were Not So Proper & Right Way! F*cking shame. > > > 2. Lack of drivers and poor guest virtualization. > > It is known that FreeBSD supports less hardware than competitors. It is, > however, a matter of popularity - the more system will have users, the > more drivers will be created by 3rd parties, so we cannot directly > affect this. It's kind of exclusive circle. > > But there is one area - virtualization - without supporting it the > FreeBSD niche will collapse very fast. It is necessary to say that it is > about guest (para-)virtualization only, not host mode. Host mode market > is already lost for FreeBSD for quite some time, may be something in the > future, but today it will waste of resources. What is needed here is > just analogue to OpenVZ (and jails/rctl already done more than half of > the work here). > > And in guest mode FreeBSD *urgently needs* working drivers/utilities for > all common suits, Citrix XenServer, VMWare vSphere (ESXi), Xen*, > Microsoft Hyper-V, etc. This must be the primary direction for > developers' forces and FreeBSD Foundation's money. We have may be about > 1 year here. Why? Because of cloud computing and VPS/VDS. It's no > matter what OS runs hoster if clients will use FreeBSD and we still have > users. I've been already asked by a customer for which I wrote > a non-portable kqueue-based daemon to rewrite for Linux because they > want to go for Amazon EC2 (it's cheaper as only used resources are > accounted) and FreeBSD there has still many in ISSUES scaring them... > > > 3. Ports and packages. > > What was the main problems with large-scale installations of FreeBSD in > that businesses? In short, that binary packages are not equal in rights > to ports, and that complicates things (i.e. requires too much work) when > one have many (> 10) servers. This was listed to me as: > > 1) No pkg and pkg-devel versions. The -devel version is headers, static > libs, programmer examples, etc. not needed in production (we could > say this part is what is actually depended on in B-deps). > > 2) Package name is dependent on options, so packages with another opts > don't work well when dependencies are rebuilt. > > 3) Conflicts: no way to have apache13 and apache22 the same time. > > 4) No dependence on base system. You may cut out something, recompile > world, deploy it on cluster and just then see that some packages are > now don't work. > > 5) Dependencies are badly designed. No version ranges in dependencies, > no alternative packages, no priorities in package search. > > 6) Update problems. The version is just coded into name of package, and > dependencies are on the entire name, so there are situations when > install/upgrade of just one package may require rebuild 3/4 of all > pkgs. You cannot easiy modify installed package without editing pkgdb > manually. It is impossible to upgrade/replace package by out of the > box tools. > > 7) Base system has no "out of the box" tools for package upgrade. Our > business opponents say this the least problem as one can always > install portupgrade, but conclude that overall base system concept > does not play well with full-featured packages (see also next part > about base system). > > 8) There is no -STABLE supported branches in ports. > > All of this could be avoided (they know about tinderbox etc.) but just > requires too much work, for their basic tasks like automated upgrade of > entire system & packages or reinstall of needed packages. > > That's problems. Next, possible (gathered) solutions. > > It is obvious that current packages are not first-class citizens, in > comparison with ports. They want ba able to run most machines without > a compiling at all (BTW, our desktop users need the same), but setting > build farms when there are many machine roles is hard. > > So packages need to be "equal in rights" in ports. The ports can have > things like this: > > .if ${OSVERSION} < 700104 || ${OSVERSION} >= 900000 > > or this: > > LIB_DEPENDS+= profiler.1:${PORTSDIR}/devel/google-perftools > > but packages are not so flexible, all you have is: > > @pkgdep perl-5.8.9_2 > > skv@ proposed the following changes: > > * OPTIONS need radio-buttons (e.g. only one of MySQL, PostgreSQL, > SQLite) and dialog(1) supports it. > > * Options must be included and installed to /var/db/ports/*/options > (this will allow to rebuild installed binary pkg as port) > > * Info about options must be included to /var/db/pkg/*/+CONTENTS like: > > @option WITH_SSL > @option WITHOUT_DEBUG > > * Dependencies must be able to specify needed OPTIONS, both required to > set and required to be unset, somewaht like: > > RUN_DEPENDS+= foobar:${PORTSDIR}/devel/foobar:+SSL:-SQLITE > > This will allow to detect conflicts with installed packages with > incompatible options. > > * For the package file names, introduce presets, e.g.: > > OPTIONS_PRESETS= default "+SSL:-DEBUG" \ > lite "-SSL:-DEBUG" > > And preset name could be put to pkg name (may be "" for default). > > * (internal) move away from CVS, rebalance to category-subcategory. > > These ideas in later discussion evolved to another additions. Let say we > are able to use multiple repositories, where "repository" is a variant > of /usr/ports tree and packages built from it. Then, each port allows to > build several packages from it, with different options. Now, if we have > a port called "softina" and user does > > pkg_add -r softina > > then dependency search must be made given needed options. So @pkgdep > consists just of "softina" and versions like ">=1.1" and options. Also, > if packages are equal in rights to ports, they need integrity/security > check. So, package file name is now like: > > softina-1.2.3_1:repo.id:1312929890.tbz # chars allowed by windows > > Here is a repository id, just a hostname like "freebsd.org" for official > ports, and an unique build id in that repo, though it were suggested > that option preset name instead of that name will be better (because > human-readable). And `pkg_add -r' fetches a single file > > softina.pkgmeta > > consisting of sections for each actual package file. Each section has > a copy of what already is in .tbz file: name, comment, options, > dependencies (and their versions and options). And one thing not present > in .tbz - it's digital signature. Fetching pkg_add looks up local key > for given repository id to check. Fetching .pkgmeta beforehand also > allows to calculate if all needed dependencies are present in repo to > not fail in the middle of 100 packages as current pkg_add may do. The > signature is in another file and optional, so user could install the > plain .tbz file manually (it still contains all needed information, > .pkgmeta is only a copy except a signature). > > The one other thing which is also optional to @pkgdep is again > repository id. This is to allow following situation: > > * company has 10 it's own internal projects > * it also has 20 modified ports from original tree > * internal pkgs depend on modified ports, not original > * it's local port tree/pkg repo may hand only those, not full tree > > Then internal pkgs may be depended on modified ports to be not > intermixed by mistake with original versions from official tree. > Repository IDs are used for that, this is optional mechanism, though. > End machines have two repositories in their config files, local and > offical, and all works correct, and local repo doesn't need to be a full > clone of ports tree. > > The problem of -devel ports could be solved by using a global knob like > WITH_DEVEL (analogue of WITHOUT_X11), and options affect parts of > pkg-plist. The solved problem: glib has perl in R-deps, just for one > script, but this script is needed only for _build_ of dependent ports. > Now, if you install irssi, you will always have glib and perl, but you > don't need perl to run irssi. And irssi could have it's build dependence > of glib:+DEVEL and run of just plain glib. > > Of course we don't want to split ports to perl, perl-base, perl-modules, > perl-doc, etc. like in Debian, and options could be solution here - one > port, several packages. Build farm may now build not one, but 2-3 common > option presets. > > Here we go to another related problem, though - ports infrastructure. > I've read 16 chapters of http://www.netbsd.org/docs/pkgsrc/ and found > several interesting moments (let's concentrate on most close sibling of > our ports, though e.g. slots in Gentoo and Arch Linux system are also > worth looking too). They already have many of what we need, and since > pkg_install/ports by Jordan Hubbard is common ancestor, it may be easier > to port from them to us needed features. > > For example, there is problem generating plist for our porters. And what > they have, a program to create port from distfile: > > | Run the program url2pkg, which will ask you for a URL. Enter > | the URL of the distribution file (in most cases a .tar.gz file) and > | watch how the basic ingredients of your package are created > | automatically. > > Just fix oddities later manually, but saves from all boilerplate work! > > Next, skv@ said about radio-buttons, and thay also have it, in two > favors: first with one from group always set, second allowin all clear: > "Exactly one of the following gecko options is required" > "At most one of the following database options may be selected" > > | PKG_OPTIONS_REQUIRED_GROUPS is like PKG_OPTIONS_OPTIONAL_GROUPS, but > | building the packages will fail if no option from the group is > | selected. PKG_OPTIONS_NONEMPTY_SETS is a list of names of sets of > | options. At least one option from each set must be selected. > > The OPTIONS, however, are handled in different way: > > $ grep "PKG.*OPTION" mk.conf > PKG_DEFAULT_OPTIONS= -arts -dvdread -esound > PKG_OPTIONS.kdebase= debug -sasl > PKG_OPTIONS.apache= suexec > > Dependencies also allow versions: > > BUILD_DEPENDS+= lua>=5.0:../../lang/lua > DEPENDS+= screen-[0-9]*:../../misc/screen > DEPENDS+= screen>=4.0:../../misc/screen > > Checksum files for packages exist on their own, and only those may be > signed not the packages itself (an alternative to .pkgmeta approach > described above). > > Another useful thing to borrow is patches/* separately from files/* > > The most visible thing in pkgsrc superior to ports, is buildlink3 > framework: > > | Buildlink is a framework in pkgsrc that controls what headers and > | libraries are seen by a package's configure and build processes. > | [...] Please note that the normal system header and library paths, > | e.g. /usr/include, /usr/lib, etc., are always searched -- buildlink3 > | is designed to insulate the package build from non-system-supplied > | software. > | [...] > | Some packages in pkgsrc install headers and libraries that coincide > | with headers and libraries present in the base system. Aside from > | a buildlink3.mk file, these packages should also include a > | builtin.mk file that includes the necessary checks to decide whether > | using the built-in software or the pkgsrc software is appropriate. > | [...] When building packages, it's possible to choose whether to set > | a global preference for using either the built-in (native) version > | or the pkgsrc version of software to satisfy a dependency. This is > | controlled by setting PREFER_PKGSRC and PREFER_NATIVE. > | PREFER_PKGSRC= yes > | PREFER_NATIVE= getopt skey tcp_wrappers > > There are more automation: > > | Up to now, the file PLIST, which contains a list of the files that > | are installed by the package, is nearly empty. Run > | bmake print-PLIST >PLIST > | to generate a probably correct list. > > Yes, they rely on their derivative of BSD make, bmake, which it itself > worth merging deifferencies to our make (for example, there is a clean > and small alternative to autotools, mk-configure, using bmake and > written in style of .include ). There may be other examples > of userland tools worth porting from NetBSD to us instead of directly > upstream, e.g. awk... but that's another story, let's continue pkg. > > Their developments of pkg_* tools contain facilities to implement a good > package manager out-of-the-box, e.g. pkg_add there has flags: > > | -A Mark package as installed automatically, as dependency of > | another package. You can use > | pkg_admin set automatic=YES > | to mark packages this way after installation, and > | pkg_admin unset automatic > | to remove the mark. If you pkg_add a package without > | specifying -A after it had already been automatically > | installed, the mark is removed. > | -U Replace an already installed version from a package. > | Implies -u. > | -u If the package that's being installed is already installed, > | an update is performed. > > These are crucial to effective binary package management. > > > 4. Base system, closely related with packages. > > The next in turn was concept of monolithic base system. The troubles > with base system were: > > 1. To change the components of the base you may only recompile it with > corresponding src.conf(5) options, but there is no track in base > system which are actually installed now. You cannot binary upgrade > a custom world, even if it is just unmodified -STABLE instead of > -RELEASE. > > 2. Consequently, there is no way to check integrity (MD5 etc.) of any > non-RELEASE variant (freebsd-update IDS is very limited). > > 3. No ties between base system and packages: who knows what previous > admin has installed, you may have compiler or may not have, etc. > Packages may silently broke if some part of base system SUDDENLY > disappears, as no dependency information is recorded. > > 4. Base system is monolithic, so there is no easy way to replace one > component with another - ports replacing base parts are hemorrhoids. > > So, they conclude, the base system concept should be eliminated and be > split to bare kernel and gazillion of packages. > > But we all know what benefits base system gives to us: it is actually > a *system* where all components are in concordance to each other, not > just a bunch of packages. Also, if it will be all split, this will > require *much more* efforts from our committers to keep everything in > sync. And our resources (efforts) are limited. > > Actually, when you face two contradictory ways, all-or-nothing, both > with pros and cons - you should select neither of them, but try to find > something which will *synthesize* both of them. Such finding is an > interesting engineering (often inventor) task. And dougb@ already said > once that between base as-is and splitting all must be something middle. > It's Cathedral vs Bazaar, after all, and our Cathedral should be updated > and still use it's cathedral benefits. After all, NetBSD has proposal of > syspkg in 2004, and still has older format - just because it is not BSD > way, something new has to be invented. > > The most obvious solution is to split base not to ~500 packages, but to, > say, ~50, and change boundaries. Why should freebsd.org developers place > boundaries in packages between themselves? So most natural solution is > to split packages out of base by the vendor criteria. Luckily, this work > is *already done* to some extent. So this will be minimal efforts > overhead, if any. > > Looking at /usr/src/sontrib and http://wiki.freebsd.org/ContribSoftware > we can identify many of what could became a package. There can be > different approaches to criteria "what is in base system": > > 1. Only what is done on freebsd.org: all contrib must go to pkgs. > > 2. Whose effective vendor is now *@*bsd.org: contrib from other BSDs may > still live, and those with ceased upstream or renamed > (non-conflicting with ports) soft like libbsdxml, too. > > 3. Axe out only the most odious contrib parts: BIND (as Peter said in > archives, host/dig could be resurrected from Attic), sendmail (could > be replaced by dma) and several others, presumably GCC/binutils & CVS > (I've also heard about painful Kereberos interferencing with Samba). > > The latter is most probable :-) given known conservatism and parts > inter-dependencies (openssl is required by freebsd-update, and it's not > *@*bsd.org - already an exception). But it still needs clear definition > of what base system is destined for. > > A philosophical observation. The more "fat" base system is, the more > things are supposed about end user (the "WHEREs" in "SELECT"), so the > more narrow target audience is. The more thin base system is, the more > popular OS can be. You elitists may sleep well: FreeBSD will never be > as popular of Linux whose "rod" is just kernel, not base sys. > > So each component of the base must be justified for the task, and for > task we may be should define to which user FreeBSD is targeted. Let it > be, say, task "the fresh setuped machine must be able to connect to > network to download and install binary packages". Then you don't need > GCC but you need editor to edit resolv.conf, less to view man pages, > tcpdump to debug network problems, send-pr to report a bug about > non-installed packages and thus a mail agent able to handle outgoing > report mail and daily periodic scripts to root while you have sex with > this (dma is suitable, sendmail is overkill). And so on. > > Axing out GCC to packages has another benefit: the newer GCC could be > used, and base could be polished to be more compiler-agnostic (hello, > clang). But this requires binary packages to have equal rights with > ports, of course. A base without a compiler is not a dramatic - you > already need some ports to do "make release", and doc team already began > to split docs to packages, that's a right direction. For POLA reasons > all axed out packages (and sendmail too, respect traditions) should be > just packaged on CD1. > > There may be another approach, not package one, but it is still in a fog > (and don't solve ports overriding base well). It's like the kernel and > modules: monolithic base system is just as monolithic kernel before > invention of modules - you the same way don't know what's in. Let's > dream a little - imagine we have our own DVCS, combining benefits of > centralized and distributed one, and not requiring splitting to multiple > repos like git, but allowing to use any given subset of files (not dirs > and subtrees as svn, *any* "inode"-based subset) from central repo > (this one "more equal" from all equal distributed repos). Such system > could be placed instead freebsd-update, binary updates of STABLE etc. > go to special branch, and user updates only those files which he have > on his system, VCS automatically tracks it. > > > 5. Too short major releases' cycle. > > I've once read a thread: > > From: mdf@FreeBSD.org > Subject: Schedule for releases > Date: Tue, 21 Dec 2010 09:47:08 -0800 > > where e.g. julian@ have said: > > | Generally a company wouldn't want to go through the pain of an OS > | upgrade more than about once in three years and often it's longer.. > | It IS a pain for them. > > And many business people reported the same. And it is known many people > are doing backports, testing etc., they should be motivated to give work > back to FreeBSD. While it is more social problem, it is also a > DVCS/bugtracker problem, but more rare major releases would still help > on it's own. > > It is known why the current scheme was adopted: the 4.11/5.3 case, > a horror. But between X.4 and X.11 there are even *several* intermediate > choices. What happens today: many conservative users (or builders of > products) consider -STABLE is really stable at the X.2 release. But just > after than an (X+1).0 is forked and all developers' attention goes to > new branch, X.3 and X.4 are not seriously developed. And due to > timelines described above, many users will then upgrade right away to > (X+2).Y, not (X+1). There are many 6.x and 7.x users in the wild, many > of 7.x users will upgrade directly to 9.x skipping 8.x. Is this good? > No. > > Aside of many branches receive not enought production testing, our > committers must do MFCs to TWO branches, stable/7 and stable/8. While > usualy having no real possibility to test properly. Nonsense. > > Is supporting two stable branches not using more efforts that it was in > 4.x times? Not so? Still could be made better. > > Proposed solution: prolonging major releases fork time a little. Just to > time so only one stable branch will exist. I hope increasing branch > lifetime to one minor release will help: last will be not .4, but .5, > and new .0 fork should occur at least after .3, not .2. We are not > Ubuntu. I understand that pace of changes is high, 3 years for each .0 > may be unacceptable, but waht about at least 2.5 years? Not like 1.8 > years currently. Rememeber, .5 and even .6 is still far from .11. > > > 6. Bug tracker, unicode and other less important trivia. > > GNATS is too old and unfriendly e.g. to user attachments. It has only > one adavantage: user is able to send report from CLI by mail without any > registration. This is essential. May be the good alternative will be RT > used at cpan.org (it also accepts by mail). Hopefully this will help > a little to solving problems. Not sure, though - and unsolved bugs are > also make users to go away from FreeBSD. > > Users also want properly working UTF-8 out-of-the-box. Not only console, > but full collation support, etc. > > There was suggestion about automatic kldloading of drivers, but this > work already began (for USB) in June - more matte of our documentation > and press relations, huh. > > Installer. Must be more featureful and more user friendly :-) This is > often may give negative first impression if installer was unable to do > something even if system after this could work well. > > Other problems, like broken cvsup, are exist, but are not critical to > Project's survival. > > > FreeBSD strengths to be concentrated on. > > The paragraph about virtualization above, as long as points below, are > roughly translated words of one small business representative in Russia, > who often acts as integrator. > > 1. BSD License. Very good for embedded vendors, we should be able to > compile and work both base and ports without licenses requiring to > open the sources. It would be good to not loose functionality without > them. > > 2. Kernel features for storage (ZFS, HAST, GEOM). We need to go to > NAS/SAN solutions niche while we have ability - it is still, because > Solaris etc. in unknown state with Oracle, BTRFS still beta. We have > about 1 year max. It would be nice to roll out "box" solution for a > failover iSCSI storage (2 PCs with ZFS raids replicated by HAST and > accessible by iSCSI with automatic role change). > > 3. Kernel features for complex network solutions (netgraph, carp, ipfw). > The niche for routers & traffic analysis is still ours. It would be > nice to take e.g. pfSense and agree with some vendor (Netgear, > D-Link, etc) to put on sale hardware with FreeBSD inside. > > So these are main ways - embedded (NAS & routers). Other market is still > not our, e.g. not an application server until there will be a killer-app > for SCTP. May be also for a DB: Solaris was first recommended for > PostgreSQL, with some efforts we may become the first. Of course, the > main way - if some commercial company will offer FreeBSD. > > We still have some time, but almost no time. We need to take decisions > right now. > > > That's all for today. Thanks to everyone who has patience to carefully > read this all entirely. > > -- > WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru > [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org > ][LJ:/nuclight] > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > -- *Pirabarlen Cheenaramen *| $3|v3n* * L'escalier, Mauritius /*memory is like prison*/ (user==selven)?free(user):user=malloc(sizeof(brain)); P Save electricity & disk space. Cat this mail to >/dev/null 2>&1 after use. From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 06:49:54 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6C93106566B for ; Tue, 23 Aug 2011 06:49:54 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 600848FC12 for ; Tue, 23 Aug 2011 06:49:54 +0000 (UTC) Received: by vws18 with SMTP id 18so6004916vws.13 for ; Mon, 22 Aug 2011 23:49:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=5DW6P6TS0iT4yyhO3K6uisR9XY4lWap9LEVAjt5OLz0=; b=dORCQ1TXQBV/kF2mWqzp6uB4CJtFWMbgFVfCiQ7D9K8wlSZ3iYEfedn6zSbk4Wg7br u7/dM5FmWVhPJ3rHz8Wz66NVtzhZ7lAhYMYFOzsBcx5dmwpV+q6HM163ePj1mvvMucXv J0hFwZentuIRpvBHD/7YpqsBqX9OHx54KoTuA= MIME-Version: 1.0 Received: by 10.52.74.2 with SMTP id p2mr3173263vdv.127.1314082192294; Mon, 22 Aug 2011 23:49:52 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.52.33.49 with HTTP; Mon, 22 Aug 2011 23:49:52 -0700 (PDT) In-Reply-To: References: Date: Tue, 23 Aug 2011 14:49:52 +0800 X-Google-Sender-Auth: -Q-gmOM_W_Gge3vEqsO5JTNdaM0 Message-ID: From: Adrian Chadd To: selven Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: vadim_nuclight@mail.ru, freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 06:49:54 -0000 And that's been my experience too. Linux users and advocates are very .. loud. FreeBSD users and advocates tend not to be so loud. Maybe that needs to change. Adrian On 23 August 2011 14:06, selven wrote: > Good post as well as pretty much scary. You mentionned about the purity o= f > FreeBSD at some point, that's one of the reasons why i prefer FreeBSD > (probably many of my friends who do work with it also), lately, with all > those ubuntu fans, trying to get the company to keep using FreeBSD has > become pretty much a hassle. Since in a technical board, with 10 people, = 7 > out of 10 are ubuntu users, with votes one always loses, and the funny th= ing > :s 7 out of 10 don't even know the difference between FreeBSD and linux, > they just voice out ubuntu because 'you will end up in trouble finding > people to maintain your stuffs if these 3 leaves' (weirdly we even fixes > there buntu bugs :s ). This user base is really a huge problem. > > On Thu, Aug 18, 2011 at 3:10 AM, Vadim Goncharov = wrote: > >> Hi, >> >> I have bad news. >> >> A month ago techlead of Rambler's Mail department declared in his blog >> that they begin migration from FreeBSD to Debian/Ubuntu. Comments to the >> blog entry (there was much flame) revealed that search department of >> Russian search engine #1 Yandex.com also plans to migrate their >> search cluster - about 30,000 servers in several DCs (~60% of total >> their servers - to Linux. The problem here is that both big companies >> were using FreeBSD from the beginning (~1997), so migration will be >> rather expensive. >> >> The official reasons (really semi-official, as these are individual >> blogs), in short, were: >> =A0* inadequate package manager and huge monolithic base system >> =A0* lack of OpenVZ-like virtualization (need CPU limiting) >> =A0* a FreeBSD marketshare forecast for 5 years >> >> Despite of several our committers working for them, the operating >> expenses for FreeBSD are considered too high by these companies. They've >> not confirmed, but the key problem is probably shortage of FreeBSD >> specialists on the market to hire (e.g. Yandex has about 20 FreeBSD >> admins and about 84 Linux admins, for all other their services and >> 40% of servers). So this is problem of FreeBSD's too low >> userbase/marketshare, caused, in turn, by other FreeBSD's problems. >> >> Given that they is just planning today, but not yet started, we have to >> solve our problems or FreeBSD will effectively die[1]. Currently >> https://ssl.netcraft.com/ssl-sample-report//CMatch/oscnt_all shows that >> still 3% of servers are running FreeBSD. We are currently probably at >> the tip of 'hype curve'[2] and then our userbase will begin to shrink, >> and the task is to solve problems, so after hard time it will rise again >> (hopefully more than it was before). >> >> =A0 [1] 'Die' means shrinking userbase size to those of NetBSD/OpenBSD. >> =A0 [2] http://www.metodolog.ru/01493/2.gif >> >> Earlier this year in this list marcel@ have said about objective view: >> http://lists.freebsd.org/pipermail/freebsd-arch/2011-January/010999.html >> from the outside, not FreeBSD people. I have posted a call to Russian >> community in my blog (http://nuclight.livejournal.com/128712.html) and >> gathered a list of problems along with possible solutions to some of >> them. The rest of this message is summary of them. >> >> In general, people sympathizing FreeBSD agree that there are no fatal >> technical problems and that FreeBSD could compete with mainstream Linux. >> That said, the 80/20 principle is in effect and 80% losed users for >> FreeBSD are caused by 20% problems. The trouble here is that what >> "outside" people view as considerable problems, we here inside FreeBSD >> view as something insignificant ("do it yourself"). Thus 80% could be >> achieved by 20% of work. I have sorted them in order of decreased >> importance (priority): >> >> =A01. Social problems of community (and marketing, docs, ...) >> =A02. Lack of drivers and virtualization. >> =A03. Ports and packages. >> =A04. Base system, closely related with packages. >> =A05. Too short major releases' cycle. >> =A06. Bug tracker, unicode and other less important trivia. >> =A0(bonus) FreeBSD strengths to be concentrated on. >> >> Now all of them more detailed. >> >> > 1. Social (psychologic) problems of community (marketing, docs, ...). >> >> This is the most important one, because all technical problems are just >> won't get solved because are even not viewed as problems. The FreeBSD >> Project does not listen to users' needs. The typical response when poor >> user want something is: "we don't need this, we won't change for you", >> with "where are your patches?" at best. Then many users go out when see >> such attitude toward them. >> >> The key points are: >> >> =A01) *The competent user is not zealot*. >> =A02) The system is *for users, not for developers*. >> >> With first: when user sees the system costs too much time for him, and >> that those problems won't get fixed and even considered such - he will >> switch to another OS. This may mean that he is able to follow our advice >> hoe to do some or another thing (e.g. to recompile all ports), but this >> is unacceptable to him because this is too much maintenance (another >> system by objectivity requires less work). Many businesses fall into >> this category. They will not with FreeBSD by any price just because they >> love OS. Unlike ourselves. >> >> With second: that's simple and not simple. It is simple in that by >> nature each person don't make a work just for himself but for all >> others, who are now "users" to him, too - just by induction that's not >> for developers only. It is not simple due to question "Why do we need >> those who don't contribute anything back to us?" which random committer >> has right to ask. The answer, in short, is: there tasks which could be >> only done by large groups of people, e.g. big corporations. For >> corporations to invest and contribute back - the system must be popular >> enough. To be popular enough means there must be a big amount of users >> who is not contributing anything. It is useful in itself, though, that >> some of those users, say 1%, will become contributors, that is, absolute >> number of our developers will also benefit from FreeBSD being much more >> popular. >> >> There are opinions in our community like: >> >> =A0| > Personally I'd like to think that that we write an OS for users. >> =A0| >> =A0| We write an OS for the people who can and will use an OS written by >> =A0| us. >> >> =A0| FreeBSD is whatever its developers make it be >> >> These are wrong and harmful nowadays. The world has changed. We have to >> admit it or we will die. We have to admit mistakes and *change* some of >> the ways we're doing things: any answer like "I don't agree, we don't >> need these users/features/etc., we *won't* do this for someone" - is >> just another step to grave. >> >> Of course that doesn't mean to do everything - we often have not enough >> time/resources to do. But that's another question and should not be >> mixed with attitude - "it's good but we have no resources, will you do?" >> vs "we don't need this, go away" (that's not only about features but >> about keeping something too). >> >> So ("system is for users") we need to (re)define who our user is. I view >> the current situation as following: >> >> =A0 =A0 =A0 100% .---------------. =A0 =A0 =A0 The stratification of ove= rall >> =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0Ubuntu =A0 =A0 | =A0 =A0 =A0 our userbas= e. Areas are: >> =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0target =A0 =A0 | 6 >> =A0 =A0 =A0 =A0 =A0 =A0| =A0 audience =A0 =A0| =A0 =A0 =A0 1 - official = developers >> =A0expand -> |---------------| >> =A0to here =A0 =A0| not our, but =A0| =A0 =A0 =A0 2 - actively participa= ting and >> =A0 =A0 =A0 =A0 =A0 =A0| =A0 competent =A0 | 5 =A0 =A0 =A0 =A0 sending p= atches, contributors >> =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 users =A0 =A0 | =A0 =A0 =A0 =A0 =A0 and= other "zealots" (whose >> =A0 =A0 =A0 =A0 =A0 =A0|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D| = =A0 =A0 =A0 =A0 =A0 "soul is with FreeBSD") >> =A0 =A0 =A0 =A0 =A0 =A0|announce@ only | 4 >> =A0 involved |---------------| =A0 =A0 =A0 3 - not so active but discuss= ing >> =A0 =A0 =A0 =A0 =A0 =A0| subscribed to | =A0 =A0 =A0 =A0 =A0 in mail lis= ts, patches go >> =A0 =A0 =A0 =A0 =A0 =A0| =A0our mailing =A0| 3 =A0 =A0 =A0 =A0 sometimes >> =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0lists =A0 =A0 =A0| >> =A0committed |- - - - - - - -| =A0 =A0 =A0 4 - busy with work to read li= sts >> =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 | 2 >> =A0 =A0 =A0 =A0 =A0 =A0|_______________| =A0 =A0 =A0 5 - can use, but co= sts are now high >> =A0 =A0 =A0 =A0 =A0 =A0| *@freebsd.org | 1 >> =A0 =A0 =A0 =A0 0% `---------------' =A0 =A0 =A0 6 - we'll never want th= em >> >> The terms "involved" and "committed" were taken from >> http://en.wikipedia.org/wiki/The_Chicken_and_the_Pig for those who are >> "at the heart" of the community, at "periphery" and all others. >> Currently, only those in (1) and partially (2) are listened to, >> sometimes with "call for ..." in (3). Meanwhile, the most important of >> our users, who are using FreeBSD in production, who are busy with real >> work in real world, are in (4) and sometimes (5). Those are largely >> ignored by the Project - even messages to announce@ last years are less >> about important events and calls for volunteers/money. >> >> The other social problem is lack of companies which offer commercial >> support of FreeBSD like RedHat does. >> >> As a possible solution to find out what user needs are, and to >> compensate the fact that not all users in (4) and (5) care enough about >> our principles and spirit, I propose a system for "weighted democracy". >> This is a voting system (I could implement it), say, there is a mail >> voting@freebsd.org to which users send filled in vote forms, with >> selected answers from a survey published in announce@. >> >> The system has a database where users are recorded given their FreeBSD >> activity in mail lists etc., and votes are summed as follows: >> >> =A0+ 1.00 vote for anyone not in database (not involved to FreeBSD) >> =A0+ a decimal logarithm of number of posts to mail lists (2 votes for 1= 00 >> =A0 posts, 3 votes for 1000 posts) >> =A0+ a binary logarithm of num PRs in GNATS db (6 votes for 64 send-pr's= ) >> =A0+ a proportion (say, N/2) of entries for this person in "svn log" (e.= g. >> =A0 in "Submitted by:") >> =A0+ an assigned (by core@) number of votes in special exceptional DB (f= or >> =A0 corporation and the like) >> >> The system then presents results for each answer: 1) how many users >> voted, 2) how many votes summed. >> >> This is by no mean to measure "exact contribution", but to defend from >> anonymous users and trolls who not care about amount of work developers >> will need to. The results are viewed as a feedback to core team, not as >> a final decision - that's all for what marketing is solely exist. There >> no adequate feedback from users to developers currently at all - >> individual posts in mail lists are too small for statistics (what about >> ministat(1) for users, huh?..). >> >> What is also in this part? Documentation. No, not in that sense. The >> observations show that while there are described solutions to problems >> in system, it's difficult for users to find it. It's somehow organized >> or misorganized that solution is not intuitive for them (e.g. someone >> migrating from Windows complained that it was difficult to find 'simply >> how to format a flash drive' in Handbook). I don't know how to handle >> this properly. May be catalogues with Best Practices howto's pointing to >> exact sections of Handbook, FAQ, articles etc.? May be better search >> mechanisms?.. >> >> And the last here about too much conservatism and rigidness in our camp. >> As one of the opponents said (roughly translated): >> >> =A0| FreeBSD, historically, was good system, let's say, "valid system >> =A0| for solid people". Pureness, strictness, etc. The base system is >> =A0| consequence of this approach. The trouble is, that in 2011 nobody >> =A0| needs pureness and strictness. Customers need transparency and fast >> =A0| reaction to their requests. If there is no transparency and speed, >> =A0| you get CentOS 6 (just released, at last). Or FreeBSD (Gentoo, >> =A0| Solaris, you name it). >> >> I don't agree that those certainly contradicts each other, but we >> definitely need to change, er, something. And don't keep something just >> because it is tradition, if that tradition has no more technical reasons >> for our users. Let's shift conservatism to another field where it is >> really needed (e.g. legacy support and other examples below). >> >> Finally, two more quotes from the arch@freebsd.org archives of last >> 2 years: >> >> =A0| From: Cyrille Lefevre >> =A0| [...] however, you answer remember me why I quit FreeBSD, cease >> =A0| fire, courtesy is the key >> >> and >> >> =A0| From: Julian H. Stacey >> =A0| Though I & many others have sent lots of send-pr, contributing even >> =A0| a spelling correction to FreeBSD is much harder than to e.g. >> =A0| http://wikipedia.org >> >> The threshold of entering to FreeBSD is too high and should be lowered. >> I can confirm the last quote on my own example, 2 months ago. I've >> submitted a patch to ipfw, adding call/return rule actions. It allows >> to organize rules in somewhat like pf anchors, and, more importantly, >> iptables chains, enabling FreeBSD ipfw to compete with Linux at least >> partially in this sphere. The patch wasn't taken in maillists, I had >> to push it in IRC, and push hard: the whole needed by many big users >> thing was deferred (and still not MFCed to 8.x) just because some >> #define's and printf's were Not So Proper & Right Way! F*cking shame. >> >> > 2. Lack of drivers and poor guest virtualization. >> >> It is known that FreeBSD supports less hardware than competitors. It is, >> however, a matter of popularity - the more system will have users, the >> more drivers will be created by 3rd parties, so we cannot directly >> affect this. It's kind of exclusive circle. >> >> But there is one area - virtualization - without supporting it the >> FreeBSD niche will collapse very fast. It is necessary to say that it is >> about guest (para-)virtualization only, not host mode. Host mode market >> is already lost for FreeBSD for quite some time, may be something in the >> future, but today it will waste of resources. What is needed here is >> just analogue to OpenVZ (and jails/rctl already done more than half of >> the work here). >> >> And in guest mode FreeBSD *urgently needs* working drivers/utilities for >> all common suits, Citrix XenServer, VMWare vSphere (ESXi), Xen*, >> Microsoft Hyper-V, etc. This must be the primary direction for >> developers' forces and FreeBSD Foundation's money. We have may be about >> 1 year here. Why? =A0Because of cloud computing and VPS/VDS. It's no >> matter what OS runs hoster if clients will use FreeBSD and we still have >> users. I've been already asked by a customer for which I wrote >> a non-portable kqueue-based daemon to rewrite for Linux because they >> want to go for Amazon EC2 (it's cheaper as only used resources are >> accounted) and FreeBSD there has still many in ISSUES scaring them... >> >> > 3. Ports and packages. >> >> What was the main problems with large-scale installations of FreeBSD in >> that businesses? In short, that binary packages are not equal in rights >> to ports, and that complicates things (i.e. requires too much work) when >> one have many (> 10) servers. This was listed to me as: >> >> 1) No pkg and pkg-devel versions. The -devel version is headers, static >> =A0 libs, programmer examples, etc. not needed in production (we could >> =A0 say this part is what is actually depended on in B-deps). >> >> 2) Package name is dependent on options, so packages with another opts >> =A0 don't work well when dependencies are rebuilt. >> >> 3) Conflicts: no way to have apache13 and apache22 the same time. >> >> 4) No dependence on base system. You may cut out something, recompile >> =A0 world, deploy it on cluster and just then see that some packages are >> =A0 now don't work. >> >> 5) Dependencies are badly designed. No version ranges in dependencies, >> =A0 no alternative packages, no priorities in package search. >> >> 6) Update problems. The version is just coded into name of package, and >> =A0 dependencies are on the entire name, so there are situations when >> =A0 install/upgrade of just one package may require rebuild 3/4 of all >> =A0 pkgs. You cannot easiy modify installed package without editing pkgd= b >> =A0 manually. It is impossible to upgrade/replace package by out of the >> =A0 box tools. >> >> 7) Base system has no "out of the box" tools for package upgrade. Our >> =A0 business opponents say this the least problem as one can always >> =A0 install portupgrade, but conclude that overall base system concept >> =A0 does not play well with full-featured packages (see also next part >> =A0 about base system). >> >> 8) There is no -STABLE supported branches in ports. >> >> All of this could be avoided (they know about tinderbox etc.) but just >> requires too much work, for their basic tasks like automated upgrade of >> entire system & packages or reinstall of needed packages. >> >> That's problems. Next, possible (gathered) solutions. >> >> It is obvious that current packages are not first-class citizens, in >> comparison with ports. They want ba able to run most machines without >> a compiling at all (BTW, our desktop users need the same), but setting >> build farms when there are many machine roles is hard. >> >> So packages need to be "equal in rights" in ports. The ports can have >> things like this: >> >> =A0.if ${OSVERSION} < 700104 || ${OSVERSION} >=3D 900000 >> >> or this: >> >> =A0LIB_DEPENDS+=3D profiler.1:${PORTSDIR}/devel/google-perftools >> >> but packages are not so flexible, all you have is: >> >> =A0@pkgdep perl-5.8.9_2 >> >> skv@ proposed the following changes: >> >> =A0* OPTIONS need radio-buttons (e.g. only one of MySQL, PostgreSQL, >> =A0 SQLite) and dialog(1) supports it. >> >> =A0* Options must be included and installed to /var/db/ports/*/options >> =A0 (this will allow to rebuild installed binary pkg as port) >> >> =A0* Info about options must be included to /var/db/pkg/*/+CONTENTS like= : >> >> =A0 =A0 @option WITH_SSL >> =A0 =A0 @option WITHOUT_DEBUG >> >> =A0* Dependencies must be able to specify needed OPTIONS, both required = to >> =A0 set and required to be unset, somewaht like: >> >> =A0 =A0 RUN_DEPENDS+=3D foobar:${PORTSDIR}/devel/foobar:+SSL:-SQLITE >> >> =A0 This will allow to detect conflicts with installed packages with >> =A0 incompatible options. >> >> =A0* For the package file names, introduce presets, e.g.: >> >> =A0 =A0 OPTIONS_PRESETS=3D default "+SSL:-DEBUG" \ >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0lite "-SSL:-DEBUG" >> >> =A0 And preset name could be put to pkg name (may be "" for default). >> >> =A0* (internal) move away from CVS, rebalance to category-subcategory. >> >> These ideas in later discussion evolved to another additions. Let say we >> are able to use multiple repositories, where "repository" is a variant >> of /usr/ports tree and packages built from it. Then, each port allows to >> build several packages from it, with different options. Now, if we have >> a port called "softina" and user does >> >> =A0pkg_add -r softina >> >> then dependency search must be made given needed options. So @pkgdep >> consists just of "softina" and versions like ">=3D1.1" and options. Also= , >> if packages are equal in rights to ports, they need integrity/security >> check. So, package file name is now like: >> >> =A0softina-1.2.3_1:repo.id:1312929890.tbz =A0 =A0# chars allowed by wind= ows >> >> Here is a repository id, just a hostname like "freebsd.org" for official >> ports, and an unique build id in that repo, though it were suggested >> that option preset name instead of that name will be better (because >> human-readable). And `pkg_add -r' fetches a single file >> >> =A0softina.pkgmeta >> >> consisting of sections for each actual package file. Each section has >> a copy of what already is in .tbz file: name, comment, options, >> dependencies (and their versions and options). And one thing not present >> in .tbz - it's digital signature. Fetching pkg_add looks up local key >> for given repository id to check. Fetching .pkgmeta beforehand also >> allows to calculate if all needed dependencies are present in repo to >> not fail in the middle of 100 packages as current pkg_add may do. The >> signature is in another file and optional, so user could install the >> plain .tbz file manually (it still contains all needed information, >> .pkgmeta is only a copy except a signature). >> >> The one other thing which is also optional to @pkgdep is again >> repository id. This is to allow following situation: >> >> =A0* company has 10 it's own internal projects >> =A0* it also has 20 modified ports from original tree >> =A0* internal pkgs depend on modified ports, not original >> =A0* it's local port tree/pkg repo may hand only those, not full tree >> >> Then internal pkgs may be depended on modified ports to be not >> intermixed by mistake with original versions from official tree. >> Repository IDs are used for that, this is optional mechanism, though. >> End machines have two repositories in their config files, local and >> offical, and all works correct, and local repo doesn't need to be a full >> clone of ports tree. >> >> The problem of -devel ports could be solved by using a global knob like >> WITH_DEVEL (analogue of WITHOUT_X11), and options affect parts of >> pkg-plist. The solved problem: glib has perl in R-deps, just for one >> script, but this script is needed only for _build_ of dependent ports. >> Now, if you install irssi, you will always have glib and perl, but you >> don't need perl to run irssi. And irssi could have it's build dependence >> of glib:+DEVEL and run of just plain glib. >> >> Of course we don't want to split ports to perl, perl-base, perl-modules, >> perl-doc, etc. like in Debian, and options could be solution here - one >> port, several packages. Build farm may now build not one, but 2-3 common >> option presets. >> >> Here we go to another related problem, though - ports infrastructure. >> I've read 16 chapters of http://www.netbsd.org/docs/pkgsrc/ and found >> several interesting moments (let's concentrate on most close sibling of >> our ports, though e.g. slots in Gentoo and Arch Linux system are also >> worth looking too). They already have many of what we need, and since >> pkg_install/ports by Jordan Hubbard is common ancestor, it may be easier >> to port from them to us needed features. >> >> For example, there is problem generating plist for our porters. And what >> they have, a program to create port from distfile: >> >> =A0| Run the program url2pkg, which will ask you for a URL. Enter >> =A0| the URL of the distribution file (in most cases a .tar.gz file) and >> =A0| watch how the basic ingredients of your package are created >> =A0| automatically. >> >> Just fix oddities later manually, but saves from all boilerplate work! >> >> Next, skv@ said about radio-buttons, and thay also have it, in two >> favors: first with one from group always set, second allowin all clear: >> =A0"Exactly one of the following gecko options is required" >> =A0"At most one of the following database options may be selected" >> >> =A0| PKG_OPTIONS_REQUIRED_GROUPS is like PKG_OPTIONS_OPTIONAL_GROUPS, bu= t >> =A0| building the packages will fail if no option from the group is >> =A0| selected. PKG_OPTIONS_NONEMPTY_SETS is a list of names of sets of >> =A0| options. At least one option from each set must be selected. >> >> The OPTIONS, however, are handled in different way: >> >> =A0$ grep "PKG.*OPTION" mk.conf >> =A0PKG_DEFAULT_OPTIONS=3D =A0 =A0-arts -dvdread -esound >> =A0PKG_OPTIONS.kdebase=3D =A0 =A0debug -sasl >> =A0PKG_OPTIONS.apache=3D =A0 =A0 suexec >> >> Dependencies also allow versions: >> >> =A0BUILD_DEPENDS+=3D lua>=3D5.0:../../lang/lua >> =A0DEPENDS+=3D =A0 =A0 =A0 screen-[0-9]*:../../misc/screen >> =A0DEPENDS+=3D =A0 =A0 =A0 screen>=3D4.0:../../misc/screen >> >> Checksum files for packages exist on their own, and only those may be >> signed not the packages itself (an alternative to .pkgmeta approach >> described above). >> >> Another useful thing to borrow is patches/* separately from files/* >> >> The most visible thing in pkgsrc superior to ports, is buildlink3 >> framework: >> >> =A0| Buildlink is a framework in pkgsrc that controls what headers and >> =A0| libraries are seen by a package's configure and build processes. >> =A0| [...] Please note that the normal system header and library paths, >> =A0| e.g. /usr/include, /usr/lib, etc., are always searched -- buildlink= 3 >> =A0| is designed to insulate the package build from non-system-supplied >> =A0| software. >> =A0| [...] >> =A0| Some packages in pkgsrc install headers and libraries that coincide >> =A0| with headers and libraries present in the base system. Aside from >> =A0| a buildlink3.mk file, these packages should also include a >> =A0| builtin.mk file that includes the necessary checks to decide whethe= r >> =A0| using the built-in software or the pkgsrc software is appropriate. >> =A0| [...] When building packages, it's possible to choose whether to se= t >> =A0| a global preference for using either the built-in (native) version >> =A0| or the pkgsrc version of software to satisfy a dependency. This is >> =A0| controlled by setting PREFER_PKGSRC and PREFER_NATIVE. >> =A0| PREFER_PKGSRC=3D =A0yes >> =A0| PREFER_NATIVE=3D =A0getopt skey tcp_wrappers >> >> There are more automation: >> >> =A0| Up to now, the file PLIST, which contains a list of the files that >> =A0| are installed by the package, is nearly empty. Run >> =A0| bmake print-PLIST >PLIST >> =A0| to generate a probably correct list. >> >> Yes, they rely on their derivative of BSD make, bmake, which it itself >> worth merging deifferencies to our make (for example, there is a clean >> and small alternative to autotools, mk-configure, using bmake and >> written in style of .include ). There may be other examples >> of userland tools worth porting from NetBSD to us instead of directly >> upstream, e.g. awk... but that's another story, let's continue pkg. >> >> Their developments of pkg_* tools contain facilities to implement a good >> package manager out-of-the-box, e.g. pkg_add there has flags: >> >> =A0| -A =A0 =A0 =A0Mark package as installed automatically, as dependenc= y of >> =A0| =A0 =A0 =A0 =A0 another package. =A0You can use >> =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pkg_admin set automatic=3DYES >> =A0| =A0 =A0 =A0 =A0 to mark packages this way after installation, and >> =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pkg_admin unset automatic >> =A0| =A0 =A0 =A0 =A0 to remove the mark. =A0If you pkg_add a package wit= hout >> =A0| =A0 =A0 =A0 =A0 specifying =A0-A after it had already been automati= cally >> =A0| =A0 =A0 =A0 =A0 installed, the mark is removed. >> =A0| -U =A0 =A0 =A0Replace an already installed version from a package. >> =A0| =A0 =A0 =A0 =A0 Implies -u. >> =A0| -u =A0 =A0 =A0If the package that's being installed is already inst= alled, >> =A0| =A0 =A0 =A0 =A0 an update is performed. >> >> These are crucial to effective binary package management. >> >> > 4. Base system, closely related with packages. >> >> The next in turn was concept of monolithic base system. The troubles >> with base system were: >> >> 1. To change the components of the base you may only recompile it with >> =A0 corresponding src.conf(5) options, but there is no track in base >> =A0 system which are actually installed now. You cannot binary upgrade >> =A0 a custom world, even if it is just unmodified -STABLE instead of >> =A0 -RELEASE. >> >> 2. Consequently, there is no way to check integrity (MD5 etc.) of any >> =A0 non-RELEASE variant (freebsd-update IDS is very limited). >> >> 3. No ties between base system and packages: who knows what previous >> =A0 admin has installed, you may have compiler or may not have, etc. >> =A0 Packages may silently broke if some part of base system SUDDENLY >> =A0 disappears, as no dependency information is recorded. >> >> 4. Base system is monolithic, so there is no easy way to replace one >> =A0 component with another - ports replacing base parts are hemorrhoids. >> >> So, they conclude, the base system concept should be eliminated and be >> split to bare kernel and gazillion of packages. >> >> But we all know what benefits base system gives to us: it is actually >> a *system* where all components are in concordance to each other, not >> just a bunch of packages. Also, if it will be all split, this will >> require *much more* efforts from our committers to keep everything in >> sync. And our resources (efforts) are limited. >> >> Actually, when you face two contradictory ways, all-or-nothing, both >> with pros and cons - you should select neither of them, but try to find >> something which will *synthesize* both of them. Such finding is an >> interesting engineering (often inventor) task. And dougb@ already said >> once that between base as-is and splitting all must be something middle. >> It's Cathedral vs Bazaar, after all, and our Cathedral should be updated >> and still use it's cathedral benefits. After all, NetBSD has proposal of >> syspkg in 2004, and still has older format - just because it is not BSD >> way, something new has to be invented. >> >> The most obvious solution is to split base not to ~500 packages, but to, >> say, ~50, and change boundaries. Why should freebsd.org developers place >> boundaries in packages between themselves? So most natural solution is >> to split packages out of base by the vendor criteria. Luckily, this work >> is *already done* to some extent. So this will be minimal efforts >> overhead, if any. >> >> Looking at /usr/src/sontrib and http://wiki.freebsd.org/ContribSoftware >> we can identify many of what could became a package. There can be >> different approaches to criteria "what is in base system": >> >> 1. Only what is done on freebsd.org: all contrib must go to pkgs. >> >> 2. Whose effective vendor is now *@*bsd.org: contrib from other BSDs may >> =A0 still live, and those with ceased upstream or renamed >> =A0 (non-conflicting with ports) soft like libbsdxml, too. >> >> 3. Axe out only the most odious contrib parts: BIND (as Peter said in >> =A0 archives, host/dig could be resurrected from Attic), sendmail (could >> =A0 be replaced by dma) and several others, presumably GCC/binutils & CV= S >> =A0 (I've also heard about painful Kereberos interferencing with Samba). >> >> The latter is most probable :-) given known conservatism and parts >> inter-dependencies (openssl is required by freebsd-update, and it's not >> *@*bsd.org - already an exception). But it still needs clear definition >> of what base system is destined for. >> >> =A0A philosophical observation. The more "fat" base system is, the more >> =A0things are supposed about end user (the "WHEREs" in "SELECT"), so the >> =A0more narrow target audience is. The more thin base system is, the mor= e >> =A0popular OS can be. You elitists may sleep well: FreeBSD will never be >> =A0as popular of Linux whose "rod" is just kernel, not base sys. >> >> So each component of the base must be justified for the task, and for >> task we may be should define to which user FreeBSD is targeted. Let it >> be, say, task "the fresh setuped machine must be able to connect to >> network to download and install binary packages". Then you don't need >> GCC but you need editor to edit resolv.conf, less to view man pages, >> tcpdump to debug network problems, send-pr to report a bug about >> non-installed packages and thus a mail agent able to handle outgoing >> report mail and daily periodic scripts to root while you have sex with >> this (dma is suitable, sendmail is overkill). And so on. >> >> Axing out GCC to packages has another benefit: the newer GCC could be >> used, and base could be polished to be more compiler-agnostic (hello, >> clang). But this requires binary packages to have equal rights with >> ports, of course. A base without a compiler is not a dramatic - you >> already need some ports to do "make release", and doc team already began >> to split docs to packages, that's a right direction. For POLA reasons >> all axed out packages (and sendmail too, respect traditions) should be >> just packaged on CD1. >> >> There may be another approach, not package one, but it is still in a fog >> (and don't solve ports overriding base well). It's like the kernel and >> modules: monolithic base system is just as monolithic kernel before >> invention of modules - you the same way don't know what's in. Let's >> dream a little - imagine we have our own DVCS, combining benefits of >> centralized and distributed one, and not requiring splitting to multiple >> repos like git, but allowing to use any given subset of files (not dirs >> and subtrees as svn, *any* "inode"-based subset) from central repo >> (this one "more equal" from all equal distributed repos). Such system >> could be placed instead freebsd-update, binary updates of STABLE etc. >> go to special branch, and user updates only those files which he have >> on his system, VCS automatically tracks it. >> >> > 5. Too short major releases' cycle. >> >> I've once read a thread: >> >> =A0From: mdf@FreeBSD.org >> =A0Subject: Schedule for releases >> =A0Date: Tue, 21 Dec 2010 09:47:08 -0800 >> >> where e.g. julian@ have said: >> >> =A0| Generally a company wouldn't want to go through the pain of an OS >> =A0| upgrade more than about once in three years and often it's longer.. >> =A0| It IS a pain for them. >> >> And many business people reported the same. And it is known many people >> are doing backports, testing etc., they should be motivated to give work >> back to FreeBSD. While it is more social problem, it is also a >> DVCS/bugtracker problem, but more rare major releases would still help >> on it's own. >> >> It is known why the current scheme was adopted: the 4.11/5.3 case, >> a horror. But between X.4 and X.11 there are even *several* intermediate >> choices. What happens today: many conservative users (or builders of >> products) consider -STABLE is really stable at the X.2 release. But just >> after than an (X+1).0 is forked and all developers' attention goes to >> new branch, X.3 and X.4 are not seriously developed. And due to >> timelines described above, many users will then upgrade right away to >> (X+2).Y, not (X+1). There are many 6.x and 7.x users in the wild, many >> of 7.x users will upgrade directly to 9.x skipping 8.x. Is this good? >> No. >> >> Aside of many branches receive not enought production testing, our >> committers must do MFCs to TWO branches, stable/7 and stable/8. While >> usualy having no real possibility to test properly. Nonsense. >> >> Is supporting two stable branches not using more efforts that it was in >> 4.x times? Not so? Still could be made better. >> >> Proposed solution: prolonging major releases fork time a little. Just to >> time so only one stable branch will exist. I hope increasing branch >> lifetime to one minor release will help: last will be not .4, but .5, >> and new .0 fork should occur at least after .3, not .2. We are not >> Ubuntu. I understand that pace of changes is high, 3 years for each .0 >> may be unacceptable, but waht about at least 2.5 years? Not like 1.8 >> years currently. Rememeber, .5 and even .6 is still far from .11. >> >> > 6. Bug tracker, unicode and other less important trivia. >> >> GNATS is too old and unfriendly e.g. to user attachments. It has only >> one adavantage: user is able to send report from CLI by mail without any >> registration. This is essential. May be the good alternative will be RT >> used at cpan.org (it also accepts by mail). Hopefully this will help >> a little to solving problems. Not sure, though - and unsolved bugs are >> also make users to go away from FreeBSD. >> >> Users also want properly working UTF-8 out-of-the-box. Not only console, >> but full collation support, etc. >> >> There was suggestion about automatic kldloading of drivers, but this >> work already began (for USB) in June - more matte of our documentation >> and press relations, huh. >> >> Installer. Must be more featureful and more user friendly :-) This is >> often may give negative first impression if installer was unable to do >> something even if system after this could work well. >> >> Other problems, like broken cvsup, are exist, but are not critical to >> Project's survival. >> >> > FreeBSD strengths to be concentrated on. >> >> The paragraph about virtualization above, as long as points below, are >> roughly translated words of one small business representative in Russia, >> who often acts as integrator. >> >> 1. BSD License. Very good for embedded vendors, we should be able to >> =A0 compile and work both base and ports without licenses requiring to >> =A0 open the sources. It would be good to not loose functionality withou= t >> =A0 them. >> >> 2. Kernel features for storage (ZFS, HAST, GEOM). We need to go to >> =A0 NAS/SAN solutions niche while we have ability - it is still, because >> =A0 Solaris etc. in unknown state with Oracle, BTRFS still beta. We have >> =A0 about 1 year max. It would be nice to roll out "box" solution for a >> =A0 failover iSCSI storage (2 PCs with ZFS raids replicated by HAST and >> =A0 accessible by iSCSI with automatic role change). >> >> 3. Kernel features for complex network solutions (netgraph, carp, ipfw). >> =A0 The niche for routers & traffic analysis is still ours. It would be >> =A0 nice to take e.g. pfSense and agree with some vendor (Netgear, >> =A0 D-Link, etc) to put on sale hardware with FreeBSD inside. >> >> So these are main ways - embedded (NAS & routers). Other market is still >> not our, e.g. not an application server until there will be a killer-app >> for SCTP. May be also for a DB: Solaris was first recommended for >> PostgreSQL, with some efforts we may become the first. Of course, the >> main way - if some commercial company will offer FreeBSD. >> >> We still have some time, but almost no time. We need to take decisions >> right now. >> >> >> That's all for today. Thanks to everyone who has patience to carefully >> read this all entirely. >> >> -- >> WBR, Vadim Goncharov. ICQ#166852181 =A0 =A0 =A0 mailto:vadim_nuclight@ma= il.ru >> [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org >> ][LJ:/nuclight] >> >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >> > > > > -- > *Pirabarlen Cheenaramen *| $3|v3n* * > L'escalier, Mauritius > > > /*memory is like prison*/ > (user=3D=3Dselven)?free(user):user=3Dmalloc(sizeof(brain)); > P Save electricity & disk space. Cat this mail to >/dev/null 2>&1 after u= se. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 07:19:54 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73D961065672 for ; Tue, 23 Aug 2011 07:19:54 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 32F6A8FC17 for ; Tue, 23 Aug 2011 07:19:54 +0000 (UTC) Received: by qyk9 with SMTP id 9so3182799qyk.13 for ; Tue, 23 Aug 2011 00:19:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FHuIKtUb/x8l9Pur1cg9YPgxjUqC4qxf9wdSLslkU2A=; b=drM/8S/I3Mvi5vnU/m7lEBBWx2e8AjZUsiM+U7LmrvFcVEBBnaYFLjbqtaXpIekv9J dkGFtJsvAvfv5283xL8vySoMvSOXwlgZUEnT0Dun0+BZ//298u2Sx0zvV9sAGRc8J7GB N+k3OfKyo+s8W5Xg5/91Qx+dhWXVTNabP9t1o= MIME-Version: 1.0 Received: by 10.224.9.14 with SMTP id j14mr1977751qaj.129.1314082475184; Mon, 22 Aug 2011 23:54:35 -0700 (PDT) Received: by 10.224.178.65 with HTTP; Mon, 22 Aug 2011 23:54:35 -0700 (PDT) In-Reply-To: References: Date: Mon, 22 Aug 2011 23:54:35 -0700 Message-ID: From: Garrett Cooper To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: vadim_nuclight@mail.ru, selven , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 07:19:54 -0000 On Mon, Aug 22, 2011 at 11:49 PM, Adrian Chadd wrote: > And that's been my experience too. > > Linux users and advocates are very .. loud. FreeBSD users and > advocates tend not to be so loud. > Maybe that needs to change. I also personally think that companies that use FreeBSD need to be louder as well. There is Isilon, iXsystems, Juniper, etc, but there are other companies that don't take a more active role in acknowledging that they run FreeBSD under the hood which only detriments FreeBSD's overall mindshare. It's funny how certain companies like Dell, IBM, etc choose to be so loud about their active Linux support -- so why should more companies using FreeBSD also not take suit? Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 12:09:19 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B13FC106566C; Tue, 23 Aug 2011 12:09:19 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 44E0D8FC0C; Tue, 23 Aug 2011 12:09:17 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA25743; Tue, 23 Aug 2011 15:09:15 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E53986B.5000804@FreeBSD.org> Date: Tue, 23 Aug 2011 15:09:15 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 12:09:19 -0000 Yes, the subject sounds quite hairy, so please let me try to explain it. First, let's consider one concrete function: static int ukbd_poll(keyboard_t *kbd, int on) { struct ukbd_softc *sc = kbd->kb_data; if (!mtx_owned(&Giant)) { /* XXX cludge */ int retval; mtx_lock(&Giant); retval = ukbd_poll(kbd, on); mtx_unlock(&Giant); return (retval); } if (on) { sc->sc_flags |= UKBD_FLAG_POLLING; sc->sc_poll_thread = curthread; } else { sc->sc_flags &= ~UKBD_FLAG_POLLING; ukbd_start_timer(sc); /* start timer */ } return (0); } This "XXX cludge" [sic] pattern is scattered around a few functions in the ukbd code and perhaps other usb code: func() { if (!mtx_owned(&Giant)) { mtx_lock(&Giant); func(); mtx_unlock(&Giant); return; } // etc ... } I have a few question directly and indirectly related to this pattern. I. [Why] do we need this pattern? Can the code be re-written in a smarter (if not to say proper) way? II. ukbd_poll() in particular can be called from the kdb context (kdb_trap -> db_trap -> db_command_loop -> etc). What would happen if the Giant is held by a thread on some other CPU (which would be hard-stopped by kdp_trap)? Won't we get a lockup here? So shouldn't this code actually check for kdb_active? III. With my stop_scheduler_on_panic patch ukbd_poll() produces infinite chains of 'infinite' recursion because mtx_owned() always returns false. This is because I skip all lock/unlock/etc operations in the post-panic context. I think that it's a good philosophical question: what operations like mtx_owned(), mtx_recursed(), mtx_trylock() 'should' return when we actually act as if no locks exist at all? My first knee-jerk reaction was to change mtx_owned() to return true in this "lock-less" context, but _hypothetically_ there could exist some symmetric code that does something like: func() { if (mtx_owned(&Giant)) { mtx_unlock(&Giant); func(); mtx_lock(&Giant); return; } // etc ... What do you think about this problem? Should we re-write ukbd_poll() (and possibly usb code) or should mutex_owned() be adjusted? That question III actually brings another thought: perhaps the whole of idea of skipping locks in a certain context is not a correct direction? Perhaps, instead we should identify all the code that could be legitimately executed in the after-panic and/or kdb contexts and make that could explicitly aware of its execution context. That is, instead of trying to make _lock, _unlock, _owned, _trylock, etc do the right thing auto-magically, we should try to make the calling code check panicstr and kdb_active and do the right thing on that level (which would include not invoking those lock-related operations or other inappropriate operations). Thank you very much in advance for your insights and help! -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 12:38:36 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F102106566B; Tue, 23 Aug 2011 12:38:36 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe04.c2i.net [212.247.154.98]) by mx1.freebsd.org (Postfix) with ESMTP id 8C2BF8FC19; Tue, 23 Aug 2011 12:38:34 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=T62/IVpqZ1mLJMkeHfndQHXgWa4ATHZu3+3wVrCy9JU= c=1 sm=1 a=SvYTsOw2Z4kA:10 a=ni9ZY7ZFbhgA:10 a=WQU8e4WWZSUA:10 a=8nJEP1OIZ-IA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=U5pupLfntSEKqH8ZPzoA:9 a=4Dju21vhlnFLU5Q-dnsA:7 a=wPNLvfGTeEIA:10 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe04.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 169339470; Tue, 23 Aug 2011 14:38:33 +0200 From: Hans Petter Selasky To: freebsd-current@freebsd.org Date: Tue, 23 Aug 2011 14:36:04 +0200 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4E53986B.5000804@FreeBSD.org> In-Reply-To: <4E53986B.5000804@FreeBSD.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108231436.04210.hselasky@c2i.net> Cc: Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 12:38:36 -0000 On Tuesday 23 August 2011 14:09:15 Andriy Gapon wrote: > Yes, the subject sounds quite hairy, so please let me try to explain it. > First, let's consider one concrete function: > > static int > ukbd_poll(keyboard_t *kbd, int on) > { > struct ukbd_softc *sc = kbd->kb_data; > > if (!mtx_owned(&Giant)) { > /* XXX cludge */ > int retval; > mtx_lock(&Giant); > retval = ukbd_poll(kbd, on); > mtx_unlock(&Giant); > return (retval); > } > > if (on) { > sc->sc_flags |= UKBD_FLAG_POLLING; > sc->sc_poll_thread = curthread; > } else { > sc->sc_flags &= ~UKBD_FLAG_POLLING; > ukbd_start_timer(sc); /* start timer */ > } > return (0); > } > > This "XXX cludge" [sic] pattern is scattered around a few functions in the > ukbd code and perhaps other usb code: > func() > { > if (!mtx_owned(&Giant)) { > mtx_lock(&Giant); > func(); > mtx_unlock(&Giant); > return; > } > > // etc ... > } > > I have a few question directly and indirectly related to this pattern. Hi Andriy, Thanks for bringing up this topic. > > I. [Why] do we need this pattern? > Can the code be re-written in a smarter (if not to say proper) way? Unless that API's surrounding ukbd change, we are stuck with auto-magic Giant locking inside ukbd. The USB stack itself does not require that Giant is used for the USB keyboard driver. It is some times since I touched this topic. From what I remember some interfacing layers of ukbd are using Giant. Some are not. Much of the existing non-USB keyboard code is written to get/put key-presses directly through legacy PCI/ISA registers. This approach is deferred in the USB stack, hence it would cause blocking delays of 1ms per polling operation. Another corner case: When you are scrolling the console having scroll lock locked, then the first printf on that console will directly call an IOCTL in the ukbd to switch off the scroll lock led. As you might be aware this can be dangerous. The ukbd is also sometimes printing stuff. The chance of deadlock and LOR is increasing. Without having to push for multiple changes outside ukbd, this problem is simply hidden in UKBD by checking if Giant is locked, which is required for starting the USB transfers in UKBD. I think it would be a very bad idea to lock another lock in a sub-function of printf. See! A multithread / taskqueue interface is needed for postponing keyboard events. And what about polling and UKBD? This needs a separate calling path I think. > > II. ukbd_poll() in particular can be called from the kdb context (kdb_trap > -> db_trap -> db_command_loop -> etc). What would happen if the Giant is > held by a thread on some other CPU (which would be hard-stopped by > kdp_trap)? Won't we get a lockup here? > So shouldn't this code actually check for kdb_active? > > III. With my stop_scheduler_on_panic patch ukbd_poll() produces infinite > chains of 'infinite' recursion because mtx_owned() always returns false. > This is because I skip all lock/unlock/etc operations in the post-panic > context. I think that it's a good philosophical question: what operations > like mtx_owned(), mtx_recursed(), mtx_trylock() 'should' return when we > actually act as if no locks exist at all? > > My first knee-jerk reaction was to change mtx_owned() to return true in > this "lock-less" context, but _hypothetically_ there could exist some > symmetric code that does something like: > func() > { > if (mtx_owned(&Giant)) { > mtx_unlock(&Giant); > func(); > mtx_lock(&Giant); > return; > } > > // etc ... > > What do you think about this problem? > Should we re-write ukbd_poll() (and possibly usb code) or should > mutex_owned() be adjusted? I think you've brought an interesing problem. I would suggest to change mtx_owned or make a new function to take an integer value to handle the case of SCHEDULER_STOPPED: mtx_owned_default(struct mtx *, int return value in case of SCHEDULER_STOPPED); > > That question III actually brings another thought: perhaps the whole of > idea of skipping locks in a certain context is not a correct direction? > Perhaps, instead we should identify all the code that could be legitimately > executed in the after-panic and/or kdb contexts and make that could > explicitly aware of its execution context. That is, instead of trying to > make _lock, _unlock, _owned, _trylock, etc do the right thing > auto-magically, we should try to make the calling code check panicstr and > kdb_active and do the right thing on that level (which would include not > invoking those lock-related operations or other inappropriate operations). > Thank you very much in advance for your insights and help! --HPS From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 13:11:10 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B90A5106564A; Tue, 23 Aug 2011 13:11:10 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 71A7C8FC0A; Tue, 23 Aug 2011 13:11:10 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id DD57D46B91; Tue, 23 Aug 2011 09:11:09 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7BFA58A02F; Tue, 23 Aug 2011 09:11:09 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 23 Aug 2011 09:11:08 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <4E53986B.5000804@FreeBSD.org> In-Reply-To: <4E53986B.5000804@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108230911.09021.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Tue, 23 Aug 2011 09:11:09 -0400 (EDT) Cc: Andriy Gapon Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 13:11:10 -0000 On Tuesday, August 23, 2011 8:09:15 am Andriy Gapon wrote: > > Yes, the subject sounds quite hairy, so please let me try to explain it. > First, let's consider one concrete function: > > static int > ukbd_poll(keyboard_t *kbd, int on) > { > struct ukbd_softc *sc = kbd->kb_data; > > if (!mtx_owned(&Giant)) { > /* XXX cludge */ > int retval; > mtx_lock(&Giant); > retval = ukbd_poll(kbd, on); > mtx_unlock(&Giant); > return (retval); > } > > if (on) { > sc->sc_flags |= UKBD_FLAG_POLLING; > sc->sc_poll_thread = curthread; > } else { > sc->sc_flags &= ~UKBD_FLAG_POLLING; > ukbd_start_timer(sc); /* start timer */ > } > return (0); > } > > This "XXX cludge" [sic] pattern is scattered around a few functions in the ukbd > code and perhaps other usb code: > func() > { > if (!mtx_owned(&Giant)) { > mtx_lock(&Giant); > func(); > mtx_unlock(&Giant); > return; > } > > // etc ... > } > > I have a few question directly and indirectly related to this pattern. > > I. [Why] do we need this pattern? > Can the code be re-written in a smarter (if not to say proper) way? Giant is recursive, it should just be always acquired. Also, this recursive call pattern is very non-obvious. A far more straightforward approach would be to just have: static int ukbd_poll(keyboard_t *kbd, int on) { struct ukbd_softc *sc = kbd->kb_data; mtx_lock(&Giant); if (on) { sc->sc_flags |= UKBD_FLAG_POLLING; sc->sc_poll_thread = curthread; } else { sc->sc_flags &= ~UKBD_FLAG_POLLING; ukbd_start_timer(sc); /* start timer */ } mtx_unlock(&Giant); return (0); } > II. ukbd_poll() in particular can be called from the kdb context (kdb_trap -> > db_trap -> db_command_loop -> etc). What would happen if the Giant is held by a > thread on some other CPU (which would be hard-stopped by kdp_trap)? Won't we get > a lockup here? > So shouldn't this code actually check for kdb_active? Probably, yes. > III. With my stop_scheduler_on_panic patch ukbd_poll() produces infinite chains > of 'infinite' recursion because mtx_owned() always returns false. This is because > I skip all lock/unlock/etc operations in the post-panic context. I think that > it's a good philosophical question: what operations like mtx_owned(), > mtx_recursed(), mtx_trylock() 'should' return when we actually act as if no locks > exist at all? Most code should not be abusing mtx_owned() in this fashion. For Giant you should just follow a simple pattern like above that lets it recurse. For your own locks you generally should use things like mtx_assert() to require all callers of a given routine to hold the lock rather than recursively acquiring the lock. Very few legitimate cases of mtx_owned() exist IMO. It is debatable if we should even have a mtx_owned() routine since we have mtx_assert(). Given this, I would leave mtx_owned() and mtx_trylock() as-is. > That question III actually brings another thought: perhaps the whole of idea of > skipping locks in a certain context is not a correct direction? > Perhaps, instead we should identify all the code that could be legitimately > executed in the after-panic and/or kdb contexts and make that could explicitly > aware of its execution context. That is, instead of trying to make _lock, > _unlock, _owned, _trylock, etc do the right thing auto-magically, we should try to > make the calling code check panicstr and kdb_active and do the right thing on that > level (which would include not invoking those lock-related operations or other > inappropriate operations). I believe that you'd end up having to maintain duplicate code for the two cases and that it would be a bigger headache as a result. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 13:29:29 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAF69106567B; Tue, 23 Aug 2011 13:29:29 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5D9B68FC0A; Tue, 23 Aug 2011 13:29:29 +0000 (UTC) Received: by qwc9 with SMTP id 9so80594qwc.13 for ; Tue, 23 Aug 2011 06:29:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=vMuZpJEjSZQCPuDfqwYLL9gKywbh9NQmhgyvWGh6ojQ=; b=YHiyW4aIN04pSQyGJV5988Rzf4AlpVgcZnHMSs7SJp1sol31IDat6zvGLmbUO04Hwt VsUKvFdZCwM1+MDGAGBfL+TXVLs06q5c388FGRMTjfIobIjIZ9LfnbcxbTrmzqu+I3aY /Hwfyb6gV2MoKcOMlMQ4CcA5j02tR/f2XDLvw= MIME-Version: 1.0 Received: by 10.229.67.65 with SMTP id q1mr2200741qci.246.1314104294902; Tue, 23 Aug 2011 05:58:14 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.98.8 with HTTP; Tue, 23 Aug 2011 05:58:14 -0700 (PDT) In-Reply-To: <4E53986B.5000804@FreeBSD.org> References: <4E53986B.5000804@FreeBSD.org> Date: Tue, 23 Aug 2011 05:58:14 -0700 X-Google-Sender-Auth: LA_e7_ZE1XEY2GweeExiwGhG3KQ Message-ID: From: mdf@FreeBSD.org To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 13:29:29 -0000 On Tue, Aug 23, 2011 at 5:09 AM, Andriy Gapon wrote: > III. =A0With my stop_scheduler_on_panic patch ukbd_poll() produces infini= te chains > of 'infinite' recursion because mtx_owned() always returns false. =A0This= is because > I skip all lock/unlock/etc operations in the post-panic context. =A0I thi= nk that > it's a good philosophical question: what operations like mtx_owned(), > mtx_recursed(), mtx_trylock() 'should' return when we actually act as if = no locks > exist at all? > > My first knee-jerk reaction was to change mtx_owned() to return true in t= his > "lock-less" context, but _hypothetically_ there could exist some symmetri= c code > that does something like: > func() > { > =A0 =A0 =A0 =A0if (mtx_owned(&Giant)) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mtx_unlock(&Giant); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0func(); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mtx_lock(&Giant); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return; > =A0 =A0 =A0 =A0} > > =A0 =A0 =A0 =A0// etc ... > > What do you think about this problem? Given that checking for a lock being held is a lot more common than checking if it's not held (in the context of mtx_assert(9) and friends), it seems reasonable to report that a lock is held in the special context of after panic. > That question III actually brings another thought: perhaps the whole of i= dea of > skipping locks in a certain context is not a correct direction? > Perhaps, instead we should identify all the code that could be legitimate= ly > executed in the after-panic and/or kdb contexts and make that could expli= citly > aware of its execution context. =A0That is, instead of trying to make _lo= ck, > _unlock, _owned, _trylock, etc do the right thing auto-magically, we shou= ld try to > make the calling code check panicstr and kdb_active and do the right thin= g on that > level (which would include not invoking those lock-related operations or = other > inappropriate operations). I don't think it's possible to identify all the code, since what runs after panic isn't tested very much. :-) I don't disagree that one should think about the issue, but providing reasonable defaults like skipping locks reduces the burden on the programmer. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 13:33:37 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5F1C1065670 for ; Tue, 23 Aug 2011 13:33:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 3ADA68FC18 for ; Tue, 23 Aug 2011 13:33:36 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA27433; Tue, 23 Aug 2011 16:33:35 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E53AC2F.1040006@FreeBSD.org> Date: Tue, 23 Aug 2011 16:33:35 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: mdf@FreeBSD.org References: <4E53986B.5000804@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 13:33:38 -0000 on 23/08/2011 15:58 mdf@FreeBSD.org said the following: > On Tue, Aug 23, 2011 at 5:09 AM, Andriy Gapon wrote: >> III. With my stop_scheduler_on_panic patch ukbd_poll() produces infinite chains >> of 'infinite' recursion because mtx_owned() always returns false. This is because >> I skip all lock/unlock/etc operations in the post-panic context. I think that >> it's a good philosophical question: what operations like mtx_owned(), >> mtx_recursed(), mtx_trylock() 'should' return when we actually act as if no locks >> exist at all? >> >> My first knee-jerk reaction was to change mtx_owned() to return true in this >> "lock-less" context, but _hypothetically_ there could exist some symmetric code >> that does something like: >> func() >> { >> if (mtx_owned(&Giant)) { >> mtx_unlock(&Giant); >> func(); >> mtx_lock(&Giant); >> return; >> } >> >> // etc ... >> >> What do you think about this problem? > > Given that checking for a lock being held is a lot more common than > checking if it's not held (in the context of mtx_assert(9) and > friends), it seems reasonable to report that a lock is held in the > special context of after panic. But, OTOH, there are snippets like this (found with grep, haven't looked at the code): /usr/src/sys/kern/kern_sx.c: while (mtx_owned(&Giant)) { >> That question III actually brings another thought: perhaps the whole of idea of >> skipping locks in a certain context is not a correct direction? >> Perhaps, instead we should identify all the code that could be legitimately >> executed in the after-panic and/or kdb contexts and make that could explicitly >> aware of its execution context. That is, instead of trying to make _lock, >> _unlock, _owned, _trylock, etc do the right thing auto-magically, we should try to >> make the calling code check panicstr and kdb_active and do the right thing on that >> level (which would include not invoking those lock-related operations or other >> inappropriate operations). > > I don't think it's possible to identify all the code, since what runs > after panic isn't tested very much. :-) I don't disagree that one > should think about the issue, but providing reasonable defaults like > skipping locks reduces the burden on the programmer. Yes, I agree with your and John's practical approach to this. Maybe print a warning about each skipped locking operation? But not sure if that would be useful, if there would be no intention of changing the code. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 14:09:03 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB26A106564A for ; Tue, 23 Aug 2011 14:09:03 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 024F28FC08 for ; Tue, 23 Aug 2011 14:09:02 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA28183 for ; Tue, 23 Aug 2011 17:09:01 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E53B47C.2000901@FreeBSD.org> Date: Tue, 23 Aug 2011 17:09:00 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: common entry point for "software" and "hardware" "panics" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 14:09:03 -0000 Too many quote signs in the subject line... let me try to explain. Currently we have two sources of detecting some trouble/inconsistency that requires a system panic/reset or debugging. One source is various checks in the program code (e.g. KASSERTs) that call panic() when a fatal inconsistency is detected. The other source is the hardware that generates a trap when something is wrong from its point of view. In this case the trap need not be a fatal one, so the software (the kernel) checks a type of trap and decides whether the condition is fatal. But let's distinguish the purely software source from the hardware+software source. Depending on the kernel options/configuration the kernel can also react in different ways to the fatal conditions. One way is to call panic(9) , the other way is to call kdb_trap. But it's even a little bit more complicated than that. So, let's consider some possibilities. !KDB, software problem: panic -> kern_reboot !KDB, fatal trap: trap -> trap_fatal -> panic -> kern_reboot KDB, !KDB_UNATTENDED, software problem: panic -> kdb_enter -> breakpoint ~> trap -> kdb_trap KDB, !KDB_UNATTENDED, fatal trap: trap -> trap_fatal -> kdb_trap Also, kdb key from console: kdb_enter -> breakpoint ~> trap -> kdb_trap panic key from console: kdb_panic -> panic -> ... and also some code calls kdb_enter instead of panic in situations that require debugging: kdb_enter -> breakpoint ~> kdb_trap So, we can see that in these examples that currently we do not have a function that would be called in all the cases. I think that it would be nice if we had some sort of a (semi-)universal front-end to panic and kdb_trap. E.g. it could be useful for some common tasks like stopping other CPUs in SMP environment. Then, it could be useful for printing some information useful in both cases like e.g. a stack trace. Or perhaps deciding whether KDB should be actually entered in a common place. Unfortunately, this is not a proposal, just sort of musings on the topic. Does anybody have some more concrete ideas here? Thank you! -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 16:39:05 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B709A106566B; Tue, 23 Aug 2011 16:39:05 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 58E328FC08; Tue, 23 Aug 2011 16:39:05 +0000 (UTC) Received: by qyk4 with SMTP id 4so2495155qyk.13 for ; Tue, 23 Aug 2011 09:39:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=K2WuxiXYc6xO80RK2nHTkfZwKjZHsbgNBhEPnUVywS8=; b=dgy54rTIDX+fgq5NSby2EqQYcbsMUKP4L79AUR98HymantcMZNXhTOwoz8MdtCsUyV y7AuvQekApaqxjWTRAc67c7d50VrbnFi1FptsKpWgjBVvqUTnbUeneO7aBoh0R/dbCXI hQ4pMKTwbrx0bE6KLWX31iNcjlVa2+ULq182U= MIME-Version: 1.0 Received: by 10.229.136.81 with SMTP id q17mr2459186qct.170.1314117544530; Tue, 23 Aug 2011 09:39:04 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.98.8 with HTTP; Tue, 23 Aug 2011 09:39:04 -0700 (PDT) In-Reply-To: <4E53B47C.2000901@FreeBSD.org> References: <4E53B47C.2000901@FreeBSD.org> Date: Tue, 23 Aug 2011 09:39:04 -0700 X-Google-Sender-Auth: IMnw2eDz2rmHjGcqDfbuoun-720 Message-ID: From: mdf@FreeBSD.org To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: common entry point for "software" and "hardware" "panics" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 16:39:05 -0000 On Tue, Aug 23, 2011 at 7:09 AM, Andriy Gapon wrote: > > Too many quote signs in the subject line... let me try to explain. > > Currently we have two sources of detecting some trouble/inconsistency tha= t > requires a system panic/reset or debugging. =A0One source is various chec= ks in the > program code (e.g. KASSERTs) that call panic() when a fatal inconsistency= is > detected. =A0The other source is the hardware that generates a trap when = something > is wrong from its point of view. =A0In this case the trap need not be a f= atal one, > so the software (the kernel) checks a type of trap and decides whether th= e > condition is fatal. =A0But let's distinguish the purely software source f= rom the > hardware+software source. > > Depending on the kernel options/configuration the kernel can also react i= n > different ways to the fatal conditions. =A0One way is to call panic(9) , = the other > way is to call kdb_trap. =A0But it's even a little bit more complicated t= han that. > > So, let's consider some possibilities. > > !KDB, software problem: > panic -> kern_reboot > > !KDB, fatal trap: > trap -> trap_fatal -> panic -> kern_reboot > > KDB, !KDB_UNATTENDED, software problem: > panic -> kdb_enter -> breakpoint ~> trap -> kdb_trap > > KDB, !KDB_UNATTENDED, fatal trap: > trap -> trap_fatal -> kdb_trap > > Also, kdb key from console: > kdb_enter -> breakpoint ~> trap -> kdb_trap > > panic key from console: > kdb_panic -> panic -> ... > > and also some code calls kdb_enter instead of panic in situations that re= quire > debugging: > kdb_enter -> breakpoint ~> kdb_trap > > So, we can see that in these examples that currently we do not have a fun= ction > that would be called in all the cases. > I think that it would be nice if we had some sort of a (semi-)universal f= ront-end > to panic and kdb_trap. =A0E.g. it could be useful for some common tasks l= ike > stopping other CPUs in SMP environment. =A0Then, it could be useful for p= rinting > some information useful in both cases like e.g. a stack trace. =A0Or perh= aps > deciding whether KDB should be actually entered in a common place. > > Unfortunately, this is not a proposal, just sort of musings on the topic. > Does anybody have some more concrete ideas here? > Thank you! I vote for the status quo. :-) That is, it seems to me that the intent behind kdb_enter() and panic() are very different. With a software fault panic is usually the right thing (since we have no way at the moment to e.g. restart the VM subsystem). debugger_on_panic then gets you a debugger if desired. kdb_enter() or breakpoint() should not be in "production" code since there may be no debugger. It seems useful to me only for intermediate debugging, and any particular use should go away when the problem is known and fixed. Cheers, matthew From owner-freebsd-arch@FreeBSD.ORG Tue Aug 23 16:42:21 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9406106564A; Tue, 23 Aug 2011 16:42:21 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4A3C38FC1A; Tue, 23 Aug 2011 16:42:21 +0000 (UTC) Received: by qwc9 with SMTP id 9so271109qwc.13 for ; Tue, 23 Aug 2011 09:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=9hHStlk0HBkpAEc/ukPQyWsOim7WnDAn4VaDH+wFkZ4=; b=FZossZt5mZvlVf5DypIKmtXyK5BNMKXYd6uXxHux/f1m/oiRmS0D9Gjx/5KP82swQf ZnCXaaA8KfKKPe7+6wg6IwN6JiOqALfoEnCiivYSALeCAEzZyvRFkljush2N6KSR4Vlq Nt3kQ9O6j9V664J7ZgLDcJEgW/eJOMgELO7wU= MIME-Version: 1.0 Received: by 10.229.136.81 with SMTP id q17mr2462715qct.170.1314117740542; Tue, 23 Aug 2011 09:42:20 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.98.8 with HTTP; Tue, 23 Aug 2011 09:42:20 -0700 (PDT) In-Reply-To: <4E53AC2F.1040006@FreeBSD.org> References: <4E53986B.5000804@FreeBSD.org> <4E53AC2F.1040006@FreeBSD.org> Date: Tue, 23 Aug 2011 09:42:20 -0700 X-Google-Sender-Auth: pNZt1sSqwDQWUgSTyGMzRq9c79I Message-ID: From: mdf@FreeBSD.org To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 16:42:21 -0000 On Tue, Aug 23, 2011 at 6:33 AM, Andriy Gapon wrote: > on 23/08/2011 15:58 mdf@FreeBSD.org said the following: >> On Tue, Aug 23, 2011 at 5:09 AM, Andriy Gapon wrote: >>> III. =A0With my stop_scheduler_on_panic patch ukbd_poll() produces infi= nite chains >>> of 'infinite' recursion because mtx_owned() always returns false. =A0Th= is is because >>> I skip all lock/unlock/etc operations in the post-panic context. =A0I t= hink that >>> it's a good philosophical question: what operations like mtx_owned(), >>> mtx_recursed(), mtx_trylock() 'should' return when we actually act as i= f no locks >>> exist at all? >>> >>> My first knee-jerk reaction was to change mtx_owned() to return true in= this >>> "lock-less" context, but _hypothetically_ there could exist some symmet= ric code >>> that does something like: >>> func() >>> { >>> =A0 =A0 =A0 =A0if (mtx_owned(&Giant)) { >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mtx_unlock(&Giant); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0func(); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mtx_lock(&Giant); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return; >>> =A0 =A0 =A0 =A0} >>> >>> =A0 =A0 =A0 =A0// etc ... >>> >>> What do you think about this problem? >> >> Given that checking for a lock being held is a lot more common than >> checking if it's not held (in the context of mtx_assert(9) and >> friends), it seems reasonable to report that a lock is held in the >> special context of after panic. > > But, OTOH, there are snippets like this (found with grep, haven't looked = at the code): > /usr/src/sys/kern/kern_sx.c: =A0 =A0 =A0 =A0 =A0 =A0while (mtx_owned(&Gia= nt)) { > >>> That question III actually brings another thought: perhaps the whole of= idea of >>> skipping locks in a certain context is not a correct direction? >>> Perhaps, instead we should identify all the code that could be legitima= tely >>> executed in the after-panic and/or kdb contexts and make that could exp= licitly >>> aware of its execution context. =A0That is, instead of trying to make _= lock, >>> _unlock, _owned, _trylock, etc do the right thing auto-magically, we sh= ould try to >>> make the calling code check panicstr and kdb_active and do the right th= ing on that >>> level (which would include not invoking those lock-related operations o= r other >>> inappropriate operations). >> >> I don't think it's possible to identify all the code, since what runs >> after panic isn't tested very much. :-) =A0I don't disagree that one >> should think about the issue, but providing reasonable defaults like >> skipping locks reduces the burden on the programmer. > > Yes, I agree with your and John's practical approach to this. > Maybe print a warning about each skipped locking operation? =A0But not su= re if that > would be useful, if there would be no intention of changing the code. Skipped locking seems like it should be left silent. I think this is a reasonable behaviour used on many operating systems -- at least I know it is used on AIX. I like the idea of a printf() in mtx_owned() since there is no completely correct answer there. Then at least on e.g. an infinite loop like you point out, it would be clear what's happening (and presumably this could use the __FILE__ and __LINE__ of the caller in the print). Cheers, matthew From owner-freebsd-arch@FreeBSD.ORG Wed Aug 24 02:19:57 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 222761065672; Wed, 24 Aug 2011 02:19:57 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id 099548FC0C; Wed, 24 Aug 2011 02:19:56 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id ED9642282A; Tue, 23 Aug 2011 20:02:21 -0600 (MDT) Received: by night.db.net (Postfix, from userid 1000) id 302295C62; Tue, 23 Aug 2011 22:02:22 -0400 (EDT) Date: Tue, 23 Aug 2011 22:02:21 -0400 From: Diane Bruce To: Adrian Chadd Message-ID: <20110824020221.GA97769@night.db.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: vadim_nuclight@mail.ru, selven , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 02:19:57 -0000 On Tue, Aug 23, 2011 at 02:49:52PM +0800, Adrian Chadd wrote: > And that's been my experience too. > > Linux users and advocates are very .. loud. FreeBSD users and > advocates tend not to be so loud. > Maybe that needs to change. I've been saying that for a while. - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db Why leave money to our children if we don't leave them the Earth? From owner-freebsd-arch@FreeBSD.ORG Wed Aug 24 16:18:54 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CB041065672 for ; Wed, 24 Aug 2011 16:18:54 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id 4C5188FC16 for ; Wed, 24 Aug 2011 16:18:54 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 4B25525D38A4; Wed, 24 Aug 2011 16:18:53 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 7DEE1BD3CD9; Wed, 24 Aug 2011 16:18:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id zZQ5lGaIwmRS; Wed, 24 Aug 2011 16:18:51 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 003F1BD3CD7; Wed, 24 Aug 2011 16:18:50 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Bjoern A. Zeeb" In-Reply-To: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> Date: Wed, 24 Aug 2011 16:18:49 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> To: George Neville-Neil X-Mailer: Apple Mail (2.1084) Cc: arch@freebsd.org Subject: Re: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 16:18:54 -0000 On Mar 19, 2011, at 6:37 AM, George Neville-Neil wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 >=20 > Howdy, >=20 > I believe it's time for us to upgrade our sysctl values for TCP = sockets so that > they are more in line with the modern world. At the moment we have = these limits on > our buffering: >=20 > kern.ipc.maxsockbuf: 262144 > net.inet.tcp.recvbuf_max: 262144 > net.inet.tcp.sendbuf_max: 262144 >=20 > I believe it's time to up these values to something that's in line = with higher speed > local networks, such as 10G. Perhaps it's time to move these to 2MB = instead of 256K. >=20 > Thoughts? This never happened, did it? Was there a reason? Index: sys/netinet/tcp_input.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/netinet/tcp_input.c (revision 225048) +++ sys/netinet/tcp_input.c (working copy) @@ -195,7 +195,7 @@ SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, recvbuf_i &VNET_NAME(tcp_autorcvbuf_inc), 0, "Incrementor step size of automatic receive buffer"); =20 -VNET_DEFINE(int, tcp_autorcvbuf_max) =3D 256*1024; +VNET_DEFINE(int, tcp_autorcvbuf_max) =3D 2*1024*1024; #define V_tcp_autorcvbuf_max VNET(tcp_autorcvbuf_max) SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, recvbuf_max, CTLFLAG_RW, &VNET_NAME(tcp_autorcvbuf_max), 0, Index: sys/netinet/tcp_output.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/netinet/tcp_output.c (revision 225048) +++ sys/netinet/tcp_output.c (working copy) @@ -117,7 +117,7 @@ SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, sendbuf_i &VNET_NAME(tcp_autosndbuf_inc), 0, "Incrementor step size of automatic send buffer"); =20 -VNET_DEFINE(int, tcp_autosndbuf_max) =3D 256*1024; +VNET_DEFINE(int, tcp_autosndbuf_max) =3D 2*1024*1024; #define V_tcp_autosndbuf_max VNET(tcp_autosndbuf_max) SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, sendbuf_max, CTLFLAG_RW, &VNET_NAME(tcp_autosndbuf_max), 0, Index: sys/sys/sockbuf.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/sys/sockbuf.h (revision 225048) +++ sys/sys/sockbuf.h (working copy) @@ -37,7 +37,7 @@ #include #include =20 -#define SB_MAX (256*1024) /* default for max chars = in sockbuf */ +#define SB_MAX (2*1024*1024) /* default for max chars = in sockbuf */ =20 /* * Constants for sb_flags field of struct sockbuf. --=20 Bjoern A. Zeeb You have to have visions! Stop bit received. Insert coin for new address family. From owner-freebsd-arch@FreeBSD.ORG Wed Aug 24 21:53:13 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B79A106566B for ; Wed, 24 Aug 2011 21:53:13 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id EFB768FC15 for ; Wed, 24 Aug 2011 21:53:12 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QwLNn-0000LP-Vt for freebsd-arch@freebsd.org; Wed, 24 Aug 2011 23:53:11 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 24 Aug 2011 23:53:11 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 24 Aug 2011 23:53:11 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Wed, 24 Aug 2011 21:52:58 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 34 Message-ID: References: <20110819154638.GA75879@over-yonder.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Matthew D. Fuller User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems/solutions: voting system & marketing surveys X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 21:53:13 -0000 Hi Matthew D. Fuller! On Fri, 19 Aug 2011 10:46:38 -0500; Matthew D. Fuller wrote about 'Re: FreeBSD problems/solutions: voting system & marketing surveys': >> In other words, Foundation must act as an intermediate party >> between those who will work and those who will pay for concrete >> tasks set up by e.g. core@ or by user suggestions/need. >> >> Why it was not done earlier? > Actually, it's my understanding that there are significant legal > issues which prevent the Foundation from acting in this way (taking > targetted donations, acting as an escrow party on specific bounties). If that is the only issue, there must be workaround for this. Circumventing such legal stoppers for good things is what lawyers are paid for. But I am from other country and don't know American law, that is task for someone living in USA. The possible solution is: a developer and company agree privately, then a company donates needed sum to Foundation, and developer signs contract with Foundation for the project as usual, just the sum is known to developer as befor. Foundation does not need to know anything about developer/company relationship here. At the very last, if that couldn't be solved with Foundation at all, it is possible to do alternative solution: a web page (subdomain of freebsd.org?) where ideas and tasks are sorted by priority, there are listed customers and volunteers for each, may be price, and of course contacts: way for them to meet and be paid outside -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Wed Aug 24 22:08:05 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA415106566B for ; Wed, 24 Aug 2011 22:08:05 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) by mx1.freebsd.org (Postfix) with ESMTP id 60B278FC21 for ; Wed, 24 Aug 2011 22:08:05 +0000 (UTC) Received: from [209.249.190.124] (helo=gnnmac.hudson-trading.com) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69) (envelope-from ) id 1QwJfH-0000pT-9J; Wed, 24 Aug 2011 16:03:07 -0400 Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=us-ascii From: George Neville-Neil In-Reply-To: Date: Wed, 24 Aug 2011 16:03:06 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <72EBCA4C-B002-4A77-9D4B-28541C1933C1@neville-neil.com> References: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> To: Bjoern A. Zeeb X-Mailer: Apple Mail (2.1244.3) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com Cc: arch@freebsd.org Subject: Re: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 22:08:05 -0000 On Aug 24, 2011, at 12:18 , Bjoern A. Zeeb wrote: > On Mar 19, 2011, at 6:37 AM, George Neville-Neil wrote: >=20 >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >>=20 >> Howdy, >>=20 >> I believe it's time for us to upgrade our sysctl values for TCP = sockets so that >> they are more in line with the modern world. At the moment we have = these limits on >> our buffering: >>=20 >> kern.ipc.maxsockbuf: 262144 >> net.inet.tcp.recvbuf_max: 262144 >> net.inet.tcp.sendbuf_max: 262144 >>=20 >> I believe it's time to up these values to something that's in line = with higher speed >> local networks, such as 10G. Perhaps it's time to move these to 2MB = instead of 256K. >>=20 >> Thoughts? >=20 >=20 > This never happened, did it? Was there a reason? >=20 I went back and looked at the mail thread. I didn't see any strong = objections so I think you should commit this for 9.x. np@ did point out that nmbclusters also lags on modern hardware so = consider upping that at the same time. Best, George > Index: sys/netinet/tcp_input.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/netinet/tcp_input.c (revision 225048) > +++ sys/netinet/tcp_input.c (working copy) > @@ -195,7 +195,7 @@ SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, recvbuf_i > &VNET_NAME(tcp_autorcvbuf_inc), 0, > "Incrementor step size of automatic receive buffer"); >=20 > -VNET_DEFINE(int, tcp_autorcvbuf_max) =3D 256*1024; > +VNET_DEFINE(int, tcp_autorcvbuf_max) =3D 2*1024*1024; > #define V_tcp_autorcvbuf_max VNET(tcp_autorcvbuf_max) > SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, recvbuf_max, CTLFLAG_RW, > &VNET_NAME(tcp_autorcvbuf_max), 0, > Index: sys/netinet/tcp_output.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/netinet/tcp_output.c (revision 225048) > +++ sys/netinet/tcp_output.c (working copy) > @@ -117,7 +117,7 @@ SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, sendbuf_i > &VNET_NAME(tcp_autosndbuf_inc), 0, > "Incrementor step size of automatic send buffer"); >=20 > -VNET_DEFINE(int, tcp_autosndbuf_max) =3D 256*1024; > +VNET_DEFINE(int, tcp_autosndbuf_max) =3D 2*1024*1024; > #define V_tcp_autosndbuf_max VNET(tcp_autosndbuf_max) > SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, sendbuf_max, CTLFLAG_RW, > &VNET_NAME(tcp_autosndbuf_max), 0, > Index: sys/sys/sockbuf.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/sys/sockbuf.h (revision 225048) > +++ sys/sys/sockbuf.h (working copy) > @@ -37,7 +37,7 @@ > #include > #include >=20 > -#define SB_MAX (256*1024) /* default for max = chars in sockbuf */ > +#define SB_MAX (2*1024*1024) /* default for max = chars in sockbuf */ >=20 > /* > * Constants for sb_flags field of struct sockbuf. >=20 > --=20 > Bjoern A. Zeeb You have to have = visions! > Stop bit received. Insert coin for new address family. >=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Aug 24 22:09:22 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB72E106564A for ; Wed, 24 Aug 2011 22:09:22 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 483B98FC14 for ; Wed, 24 Aug 2011 22:09:22 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QwLdR-0007PW-7K for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 00:09:21 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 00:09:21 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 00:09:21 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Wed, 24 Aug 2011 22:09:07 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 64 Message-ID: References: <705869186.20110819012421@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Robert Watson User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 22:09:22 -0000 Hi Robert Watson! On Fri, 19 Aug 2011 09:23:46 +0100 (BST); Robert Watson wrote about 'Re: FreeBSD problems and preliminary ways to solve': >>> everything is ``suitable'' for common tasks. And it is NOT ENOUGH >>> to be technically better. System should be far more superior to be >>> chosen, if it is not fancy/trendy. Yes, I belive, that FreeBSD is >>> better than Linux (at least on supported hardware) in server tasks, >>> more clear, more solid, etc. But it is ``only'' better, and is not >>> enough. >> >> No. The whole message was about that FreeBSD is worse in several areas, >> which masks out areas where it is better. If that will get fixed, we could >> talk about is it needing to be "far more" superior or not. But not before >> that as excuse to do nothing - no such excuse exists. > I'd point out that the key thing here is to produce, distribute, and as you > rightly point out *make easy*, new technologies that are transformative in a > way that makes FreeBSD compelling. So compelling that you'd rather switch OS > than not have it. Totally agreed. And: > has never been done before, giving security that has never been had before. > This required a new solution (although if you read our USENIX Security or > forthcoming CACM paper, it is grounded in some quite old but promising ideas) > that you can't find anywhere else. [...] > something that just isn't plausibe with other > existing sandboxing technologies. Obviously, we hope that the rest of the > world will adopt our APIs (and have spotted the OpenBSD folk working on this > already, and there's a Linux port out of Google), but I hope for a bit of a > run where you have to come to us to get this! And being the place APIs and > ideas like this come from is important. Absolutely right. I'll say more: the strength comes from having unique useful features which are pioneered here, actually new to industry and born here. Don't borrow from competitors but look at them and invent something better. I can give an example in network stack: while there are voices to adapt for us Linux' Netlink, we should get more usage of Netgraph here, rather adapting and being always second (looser, outsider). I've always liked FreeBSD for organicly synthesizing something new and "quite old but promising ideas". > It's worth observing that the success of Linux (and FreeBSD) to date comes out > of a few very basic but fundamental improvements in OS design: > - Tightly integrated networking > - Much cheaper than any competition > - Extremely stable compared to the competition There is one hidden, more fundamental fact: a compettion itself. The competition makes industry to innovate, and there are more innovations happening here (Linux and FreeBSD) than in Windows... or at least it looks so, thus being more attractive to users. > walking all over the competition. Buzz is a critical part of selling ideas in > open source (for better or worse), and there's no reason we can't play in that > game a bit while maintaining our boring and staid personalities :-). Sure. And taking surveys into account, we could just simply summarize: FreeBSD needs marketing :-) -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Wed Aug 24 22:16:31 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C0C2106566C for ; Wed, 24 Aug 2011 22:16:31 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 46A6C8FC12 for ; Wed, 24 Aug 2011 22:16:31 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QwLkM-00022F-D7 for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 00:16:30 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 00:16:30 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 00:16:30 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Wed, 24 Aug 2011 22:16:18 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 28 Message-ID: References: <810527321.20110819123700@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Lev Serebryakov User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 22:16:31 -0000 Hi Lev Serebryakov! On Fri, 19 Aug 2011 12:37:00 +0400; Lev Serebryakov wrote about 'Re: FreeBSD problems and preliminary ways to solve': >> 3. Kernel features for complex network solutions (netgraph, carp, ipfw). >> The niche for routers & traffic analysis is still ours. It would be >> nice to take e.g. pfSense and agree with some vendor (Netgear, >> D-Link, etc) to put on sale hardware with FreeBSD inside. > What about 10G routing? Here are reports about full-bandwidth 10G routing on modern > Intel NICs with Linux (and multi-core server), but I didn't see any > such data for FreeBSD, and somebody says, that Intel drivers and > network stack is not so good parallel in FreeBSD. "Too narrow is their circle, so terribly far they are from people". The 10G routing is definetely not a critical FreeBSD problem, actual for broad user masses. What percentage of even Linux users do have it? I am currently working as admin of small Linux HPC cluster - MPI, Infiniband, you know, 10G/40G. That area is unknown to average Linux user: all that they know is that Linux is used on such computers (Top-500), but that's all, just a bit of proudness, not something real. When they come to me try to use it there is little help for them that's Linux too :-) Everyday needs are very different. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Wed Aug 24 22:25:07 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A33F106566B for ; Wed, 24 Aug 2011 22:25:07 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id D876B8FC0C for ; Wed, 24 Aug 2011 22:25:06 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QwLsf-0005GA-3h for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 00:25:05 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 00:25:05 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 00:25:05 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Wed, 24 Aug 2011 22:20:40 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 26 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Marcin Wisnicki User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 22:25:07 -0000 Hi Marcin Wisnicki! On Mon, 22 Aug 2011 17:04:54 +0000 (UTC); Marcin Wisnicki wrote about 'Re: FreeBSD problems and preliminary ways to solve': >>> * Options must be included and installed to /var/db/ports/*/options >>> (this will allow to rebuild installed binary pkg as port) >>> >>> >> This could be a good idea :) I'll keep it for pkgng > See also +BUILD_INFO and other metadata files in pkgsrc. > I couldn't find documentation but you can grab random binary package and > inspect its contents. > > It's a shame that pkgsrc is so often ignored here. It solves so many > problems of ports and is so far ahead that any effort to improve ports > will never match what is already available from pkgsrc. > That's even more strange given that early years, pkg_* tools in NetBSD (pkgsrc) and FreeBSD got active code exchange. Why has it stopped later?.. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Wed Aug 24 22:30:24 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7716106566B for ; Wed, 24 Aug 2011 22:30:24 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 108FE8FC19 for ; Wed, 24 Aug 2011 22:30:22 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QwLxl-0007Hl-5i for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 00:30:21 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 00:30:21 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 00:30:21 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Wed, 24 Aug 2011 22:30:09 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 28 Message-ID: References: <1313785806.56747.YahooMailClassic@web113503.mail.gq1.yahoo.com> <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Rick Macklem User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 22:30:24 -0000 Hi Rick Macklem! On Fri, 19 Aug 2011 18:38:35 -0400 (EDT); Rick Macklem wrote about 'Re: FreeBSD problems and preliminary ways to solve': > One thing I thought I'd bring up (since I haven't seen it > mentioned yet) is Debian GNU/kFreeBSD. I haven't tried it, > so I'm talking through my hat a bunch, but... > It seems to me that FreeBSD should do what it can to support > this effort. Why? Well, I suspect a lot of why organizations No. FreeBSD has limited resources. We need to: a) fix our own problems b) develop our own unique features (see my message to Robert Watson) And supporting Debian will mean wasting resources which could be spent to these. This will effectively kill FreeBSD as a separate entity if our problems will stay unfixed and get worse due to this. Leave this work to Debian: they already have a wide community and many resources. Any Linux distro has more resources than BSD because they just only pack someone's software, and we have to also actually *develop* those software (most of all, kernel and libc). -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 00:17:23 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3A7B106566C for ; Thu, 25 Aug 2011 00:17:23 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id 0C1558FC18 for ; Thu, 25 Aug 2011 00:17:22 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 4CF6A25D37C0; Thu, 25 Aug 2011 00:17:21 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 77EC2BD3CEB; Thu, 25 Aug 2011 00:17:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id ahs9oo7nGj8i; Thu, 25 Aug 2011 00:17:18 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 355E1BD3C1C; Thu, 25 Aug 2011 00:17:17 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Bjoern A. Zeeb" In-Reply-To: <72EBCA4C-B002-4A77-9D4B-28541C1933C1@neville-neil.com> Date: Thu, 25 Aug 2011 00:17:16 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <14F25601-6F2A-40F7-B4C7-82139ECE7998@lists.zabbadoz.net> References: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> <72EBCA4C-B002-4A77-9D4B-28541C1933C1@neville-neil.com> To: George Neville-Neil X-Mailer: Apple Mail (2.1084) Cc: arch@freebsd.org Subject: Re: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 00:17:23 -0000 On Aug 24, 2011, at 8:03 PM, George Neville-Neil wrote: >=20 > On Aug 24, 2011, at 12:18 , Bjoern A. Zeeb wrote: >=20 >> On Mar 19, 2011, at 6:37 AM, George Neville-Neil wrote: >>=20 >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>>=20 >>> Howdy, >>>=20 >>> I believe it's time for us to upgrade our sysctl values for TCP = sockets so that >>> they are more in line with the modern world. At the moment we have = these limits on >>> our buffering: >>>=20 >>> kern.ipc.maxsockbuf: 262144 >>> net.inet.tcp.recvbuf_max: 262144 >>> net.inet.tcp.sendbuf_max: 262144 >>>=20 >>> I believe it's time to up these values to something that's in line = with higher speed >>> local networks, such as 10G. Perhaps it's time to move these to 2MB = instead of 256K. >>>=20 >>> Thoughts? >>=20 >>=20 >> This never happened, did it? Was there a reason? >>=20 >=20 > I went back and looked at the mail thread. I didn't see any strong = objections > so I think you should commit this for 9.x. >=20 > np@ did point out that nmbclusters also lags on modern hardware so = consider upping > that at the same time. The below seems to need a cast for max_sb_adj which is already present = for the sysctl; currently doing a universe and will send it to re@. = Thanks for the reply. nmbclusters is harder to change as it's not just a max but directly = needs/eats memory so we cannot just up it; we'll have to look into = auto-tuning after 9 was branched and can possibly merge it for 9.1 or = so. /bz >> Index: sys/netinet/tcp_input.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/netinet/tcp_input.c (revision 225048) >> +++ sys/netinet/tcp_input.c (working copy) >> @@ -195,7 +195,7 @@ SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, = recvbuf_i >> &VNET_NAME(tcp_autorcvbuf_inc), 0, >> "Incrementor step size of automatic receive buffer"); >>=20 >> -VNET_DEFINE(int, tcp_autorcvbuf_max) =3D 256*1024; >> +VNET_DEFINE(int, tcp_autorcvbuf_max) =3D 2*1024*1024; >> #define V_tcp_autorcvbuf_max VNET(tcp_autorcvbuf_max) >> SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, recvbuf_max, CTLFLAG_RW, >> &VNET_NAME(tcp_autorcvbuf_max), 0, >> Index: sys/netinet/tcp_output.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/netinet/tcp_output.c (revision 225048) >> +++ sys/netinet/tcp_output.c (working copy) >> @@ -117,7 +117,7 @@ SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, = sendbuf_i >> &VNET_NAME(tcp_autosndbuf_inc), 0, >> "Incrementor step size of automatic send buffer"); >>=20 >> -VNET_DEFINE(int, tcp_autosndbuf_max) =3D 256*1024; >> +VNET_DEFINE(int, tcp_autosndbuf_max) =3D 2*1024*1024; >> #define V_tcp_autosndbuf_max VNET(tcp_autosndbuf_max) >> SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, sendbuf_max, CTLFLAG_RW, >> &VNET_NAME(tcp_autosndbuf_max), 0, >> Index: sys/sys/sockbuf.h >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/sys/sockbuf.h (revision 225048) >> +++ sys/sys/sockbuf.h (working copy) >> @@ -37,7 +37,7 @@ >> #include >> #include >>=20 >> -#define SB_MAX (256*1024) /* default for max = chars in sockbuf */ >> +#define SB_MAX (2*1024*1024) /* default for max = chars in sockbuf */ >>=20 >> /* >> * Constants for sb_flags field of struct sockbuf. >>=20 >> --=20 >> Bjoern A. Zeeb You have to have = visions! >> Stop bit received. Insert coin for new address family. >>=20 >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" --=20 Bjoern A. Zeeb You have to have visions! Stop bit received. Insert coin for new address family. From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 02:55:42 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D45B6106564A for ; Thu, 25 Aug 2011 02:55:42 +0000 (UTC) (envelope-from milo@cyberlifelabs.com) Received: from qmta04.westchester.pa.mail.comcast.net (qmta04.westchester.pa.mail.comcast.net [76.96.62.40]) by mx1.freebsd.org (Postfix) with ESMTP id 953358FC0A for ; Thu, 25 Aug 2011 02:55:42 +0000 (UTC) Received: from omta03.westchester.pa.mail.comcast.net ([76.96.62.27]) by qmta04.westchester.pa.mail.comcast.net with comcast id QShh1h00A0bG4ec54SiThn; Thu, 25 Aug 2011 02:42:27 +0000 Received: from [10.0.1.101] ([71.227.219.222]) by omta03.westchester.pa.mail.comcast.net with comcast id QSiR1h01X4oVxvK3PSiSPS; Thu, 25 Aug 2011 02:42:27 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1084) From: Milo Hyson In-Reply-To: Date: Wed, 24 Aug 2011 19:42:23 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <04EEADEE-380F-48A0-BBBF-1A1673228F90@cyberlifelabs.com> References: <705869186.20110819012421@serebryakov.spb.ru> To: freebsd-arch@freebsd.org X-Mailer: Apple Mail (2.1084) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 02:55:42 -0000 On Aug 24, 2011, at 3:09 PM, Vadim Goncharov wrote: >> walking all over the competition. Buzz is a critical part of selling = ideas in=20 >> open source (for better or worse), and there's no reason we can't = play in that=20 >> game a bit while maintaining our boring and staid personalities :-). >=20 > Sure. And taking surveys into account, we could just simply summarize: > FreeBSD needs marketing :-) That begs the question of to whom FreeBSD should be marketed. Home = users? Small-office admins? Datacenter admins? Embedded developers? - Milo Hyson Chief Scientist CyberLife Labs, Inc. From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 04:01:00 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD6E31065670 for ; Thu, 25 Aug 2011 04:01:00 +0000 (UTC) (envelope-from alex-goncharov@comcast.net) Received: from qmta14.westchester.pa.mail.comcast.net (qmta14.westchester.pa.mail.comcast.net [76.96.59.212]) by mx1.freebsd.org (Postfix) with ESMTP id 99B8F8FC14 for ; Thu, 25 Aug 2011 04:01:00 +0000 (UTC) Received: from omta17.westchester.pa.mail.comcast.net ([76.96.62.89]) by qmta14.westchester.pa.mail.comcast.net with comcast id QTdt1h0071vXlb85ETnlWT; Thu, 25 Aug 2011 03:47:45 +0000 Received: from hanssachs.home ([24.61.85.144]) by omta17.westchester.pa.mail.comcast.net with comcast id QTnj1h00d36qgMk3dTnk5Y; Thu, 25 Aug 2011 03:47:44 +0000 Received: from algo by hanssachs.home with local (Exim 4.76 (FreeBSD)) (envelope-from ) id 1QwQus-000256-ME for freebsd-arch@freebsd.org; Wed, 24 Aug 2011 23:47:42 -0400 From: Alex Goncharov To: freebsd-arch@freebsd.org In-reply-to: <04EEADEE-380F-48A0-BBBF-1A1673228F90@cyberlifelabs.com> (message from Milo Hyson on Wed, 24 Aug 2011 19:42:23 -0700) References: <705869186.20110819012421@serebryakov.spb.ru> <04EEADEE-380F-48A0-BBBF-1A1673228F90@cyberlifelabs.com> Message-Id: Sender: Alex Goncharov Date: Wed, 24 Aug 2011 23:47:42 -0400 Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Alex Goncharov List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 04:01:00 -0000 ,--- Milo Hyson (Wed, 24 Aug 2011 19:42:23 -0700) ----* | On Aug 24, 2011, at 3:09 PM, Vadim Goncharov wrote: | >> walking all over the competition. Buzz is a critical part of selling ideas in | >> open source (for better or worse), and there's no reason we can't play in that | >> game a bit while maintaining our boring and staid personalities :-). | > | > Sure. And taking surveys into account, we could just simply summarize: | > FreeBSD needs marketing :-) | | That begs the question of to whom FreeBSD should be marketed. Home | users? Small-office admins? Datacenter admins? Embedded developers? Horror if a wrong decision is made here, and some critical user group is under-marketed: the ideas will no longer sell, the volunteers stop volunteering, and we, the freeloaders, stop freeloading. A company in Russia starts using the free (imagine, they won't pay for it!) Linux instead of FreeBSD: one more ports user is lost, oh my!... Let's unite and find a company that backs FreeBSD commercially: it'll surely make a good profit... doesn't matter if not, when the FreeBSD fate is at stake... we only have a year left... Gotta win that competition -- do something, quick, y'all!.. -- Alex -- alex-goncharov@comcast.net -- From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 04:01:14 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D51011065675 for ; Thu, 25 Aug 2011 04:01:14 +0000 (UTC) (envelope-from gnuyoga@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 6C7AC8FC0A for ; Thu, 25 Aug 2011 04:01:14 +0000 (UTC) Received: by wwi36 with SMTP id 36so1917435wwi.31 for ; Wed, 24 Aug 2011 21:01:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=x-rim-org-msg-ref-id:message-id:content-transfer-encoding:reply-to :x-priority:sensitivity:importance:subject:to:from:date:content-type :mime-version; bh=+TnnBn3vAeIM3HiRLl4coVgowsbVGmpd8izcWseGkAE=; b=nJUBIkPPA2zEEcvrdGDyQgsYmZldR/GkTTF7eJ4d0U54mCDfuXCoc9ozbOSIx4qYs7 WGLLbavySGOel6sZcTYfbXuODHtqfkjOHMPeRm9KJZsoq0fog7OHYpv3KhCskC3AgG9x vPxEP8cXu7z28FpB0xpUxgmoCMCtjC3KXNyDg= Received: by 10.227.113.130 with SMTP id a2mr44129wbq.72.1314243261294; Wed, 24 Aug 2011 20:34:21 -0700 (PDT) Received: from 172.18.208.219 (bda-178-239-86-97.bis7.eu.blackberry.com [178.239.86.97]) by mx.google.com with ESMTPS id gb10sm102516wbb.22.2011.08.24.20.34.18 (version=SSLv3 cipher=OTHER); Wed, 24 Aug 2011 20:34:19 -0700 (PDT) X-rim-org-msg-ref-id: 35765857 Message-ID: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Content-Transfer-Encoding: base64 X-Priority: Normal Sensitivity: Normal Importance: Normal To: "Milo Hyson" , owner-freebsd-arch@freebsd.org, freebsd-arch@freebsd.org From: gnuyoga@gmail.com Date: Thu, 25 Aug 2011 03:34:15 +0000 Content-Type: text/plain; charset="Windows-1252" MIME-Version: 1.0 X-Mailman-Approved-At: Thu, 25 Aug 2011 04:24:30 +0000 Cc: Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gnuyoga@gmail.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 04:01:14 -0000 PiANCj4gU3VyZS4gQW5kIHRha2luZyBzdXJ2ZXlzIGludG8gYWNjb3VudCwgd2UgY291bGQganVz dCBzaW1wbHkgc3VtbWFyaXplOg0KPiBGcmVlQlNEIG5lZWRzIG1hcmtldGluZyA6LSkNCg0KVGhh dCBiZWdzIHRoZSBxdWVzdGlvbiBvZiB0byB3aG9tIEZyZWVCU0Qgc2hvdWxkIGJlIG1hcmtldGVk LiBIb21lIHVzZXJzPyBTbWFsbC1vZmZpY2UgYWRtaW5zPyBEYXRhY2VudGVyIGFkbWlucz8gRW1i ZWRkZWQgZGV2ZWxvcGVycz8NCg0KDQpQZXJoYXBzIHdlIGNhbiBzdGFydCBmcm9tIHdoYXQgaXMg Y3VycmVudGx5IHVzZWQvZGVwbG95ZWQuIEVhc2llciB0byBzdGFydCBmcm9tIHdoYXQgd2UgaGF2 ZSBkb25lIHRoYW4gZmlndXJpbmcgb3V0IHdoYXQgYWxsIGl0IGNhbiBkby4gV2hhdCBzYXkgPw0K DQoNClNlbnQgZnJvbSBCbGFja0JlcnJ5riBvbiBBaXJ0ZWw= From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 05:17:44 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B65C31065672 for ; Thu, 25 Aug 2011 05:17:44 +0000 (UTC) (envelope-from milo@cyberlifelabs.com) Received: from qmta04.emeryville.ca.mail.comcast.net (qmta04.emeryville.ca.mail.comcast.net [76.96.30.40]) by mx1.freebsd.org (Postfix) with ESMTP id 9FD558FC21 for ; Thu, 25 Aug 2011 05:17:44 +0000 (UTC) Received: from omta12.emeryville.ca.mail.comcast.net ([76.96.30.44]) by qmta04.emeryville.ca.mail.comcast.net with comcast id QV0G1h0010x6nqcA4V4WA5; Thu, 25 Aug 2011 05:04:30 +0000 Received: from [10.0.1.101] ([71.227.219.222]) by omta12.emeryville.ca.mail.comcast.net with comcast id QV4V1h00S4oVxvK8YV4Wsz; Thu, 25 Aug 2011 05:04:30 +0000 Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Milo Hyson X-Priority: Normal In-Reply-To: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Date: Wed, 24 Aug 2011 22:04:33 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <5E028F46-D0CA-4F4D-B59A-0FF524FFC4F5@cyberlifelabs.com> References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> To: freebsd-arch@freebsd.org X-Mailer: Apple Mail (2.1084) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 05:17:44 -0000 On Aug 24, 2011, at 8:34 PM, gnuyoga@gmail.com wrote: >>=20 >> Sure. And taking surveys into account, we could just simply = summarize: >> FreeBSD needs marketing :-) >=20 > That begs the question of to whom FreeBSD should be marketed. Home = users? Small-office admins? Datacenter admins? Embedded developers? >=20 >=20 > Perhaps we can start from what is currently used/deployed. Easier to = start from what we have done than figuring out what all it can do. What = say ? There's no need to figure it all out in advance. Targeting one or two = top-priority demographics is a perfectly reasonable start. What is it = that makes FreeBSD better than other options? Who would benefit most = from that? Start with them. - Milo Hyson Chief Scientist CyberLife Labs, Inc. From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 05:24:56 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 957CB106566C; Thu, 25 Aug 2011 05:24:56 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 421658FC0A; Thu, 25 Aug 2011 05:24:55 +0000 (UTC) Received: by yxn22 with SMTP id 22so1701202yxn.13 for ; Wed, 24 Aug 2011 22:24:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=NYmZYLT/0MGwSKUEd7tHydc3cF5nZxh9TpxYvSqk46M=; b=cdk6KfFs7Mrdysg+aUdGBVj/SlOh1TX+VikmkJcutQ9Y3dkw1gy/w65krc6Rdzmw6k UNzmJteoVjC8sHOccpA6x2LBqqi3wstCAAsjEEOpbGm/28Gk+Dq+CDe/z8oh7jnTDmHU szRKRvHjtzp3txc26EgDbaf3vbkUs5WPX58Rw= MIME-Version: 1.0 Received: by 10.150.32.14 with SMTP id f14mr384581ybf.201.1314249894635; Wed, 24 Aug 2011 22:24:54 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.145.21 with HTTP; Wed, 24 Aug 2011 22:24:54 -0700 (PDT) In-Reply-To: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Date: Thu, 25 Aug 2011 13:24:54 +0800 X-Google-Sender-Auth: 6qbDkW-0Disd8iAhH8S8OLgKKv8 Message-ID: From: Adrian Chadd To: gnuyoga@gmail.com Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch@freebsd.org, Milo Hyson , owner-freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 05:24:56 -0000 On 25 August 2011 11:34, wrote: >> >> Sure. And taking surveys into account, we could just simply summarize: >> FreeBSD needs marketing :-) > > That begs the question of to whom FreeBSD should be marketed. Home users? Small-office admins? Datacenter admins? Embedded developers? > > Perhaps we can start from what is currently used/deployed. Easier to start from what we have done than figuring out what all it can do. What say ? Pick your area of interest. Work on making it more useful. Be very loud and vocal about what you're doing. Explain how it's better than the alternatives. The project as a whole may not necessarily need a "project dictator" per se. It doesn't need to figure out who FreeBSD needs to be marketed to. What _I_ think the project needs is louder developers and advocates; easier install/management tools (especially for VM/cluster management ; ports is a pain in the ass as viewed by a lot of people - who think RPM/DEB is fine (and have built large networks around such)) and some more use cases where FreeBSD makes sense. Right now Linux runs in most places, it may not be "awesome" but it's "good enough", and the entry barrier is sufficiently low for people to bootstrap a lot of new stuff on it. Combine that with increasing numbers of users and developers who know Linux only and recommend it over anything else, regardless of technical merit. So, let's stop talking about it; pick something you think should be better, run with it, and be very vocal about what you do. Adrian From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 10:37:56 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9106E1065670; Thu, 25 Aug 2011 10:37:56 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe06.c2i.net [212.247.154.162]) by mx1.freebsd.org (Postfix) with ESMTP id ACFBD8FC08; Thu, 25 Aug 2011 10:37:55 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=RCcoR6dhO4r2plSBAg1vtKNWl/yGWcPYGWoK9vAZDcU= c=1 sm=1 a=SvYTsOw2Z4kA:10 a=ni9ZY7ZFbhgA:10 a=WQU8e4WWZSUA:10 a=8nJEP1OIZ-IA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=Z3Q73qxEUil-ZZdhi3kA:9 a=lmfJouIePbVyiHv-sIwA:7 a=wPNLvfGTeEIA:10 a=lu8jnEJfmK6Ue0LV:21 a=hvKIPJzv4vDNgfuC:21 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe06.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 169798292; Thu, 25 Aug 2011 12:37:46 +0200 From: Hans Petter Selasky To: freebsd-arch@freebsd.org Date: Thu, 25 Aug 2011 12:35:15 +0200 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> In-Reply-To: <201108230911.09021.jhb@freebsd.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108251235.15853.hselasky@c2i.net> Cc: Andriy Gapon Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 10:37:56 -0000 On Tuesday 23 August 2011 15:11:08 John Baldwin wrote: > > I. [Why] do we need this pattern? > > Can the code be re-written in a smarter (if not to say proper) way? > Hi, > Giant is recursive, it should just be always acquired. Also, this > recursive call pattern is very non-obvious. A far more straightforward > approach would be to just have: Unless Witness is changed, that won't work. It will lead to LOR warnings I think. Imagine that the root function locks Giant, then another lock is locked and then ukbd_poll() is called. Won't the second lock of Giant cause a LOR warning? > > static int > ukbd_poll(keyboard_t *kbd, int on) > { > struct ukbd_softc *sc = kbd->kb_data; > > mtx_lock(&Giant); > if (on) { > sc->sc_flags |= UKBD_FLAG_POLLING; > sc->sc_poll_thread = curthread; > } else { > sc->sc_flags &= ~UKBD_FLAG_POLLING; > ukbd_start_timer(sc); /* start timer */ > } > mtx_unlock(&Giant); > return (0); > } > > Most code should not be abusing mtx_owned() in this fashion. For Giant you > should just follow a simple pattern like above that lets it recurse. For > your own locks you generally should use things like mtx_assert() to > require all callers of a given routine to hold the lock rather than > recursively acquiring the lock. Very few legitimate cases of mtx_owned() > exist IMO. It is debatable if we should even have a mtx_owned() routine > since we have mtx_assert(). How would you solve the LOR case? --HPS From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 11:34:50 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A248F106566C; Thu, 25 Aug 2011 11:34:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C0BBD8FC0A; Thu, 25 Aug 2011 11:34:49 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA02587; Thu, 25 Aug 2011 14:34:45 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E563354.5020106@FreeBSD.org> Date: Thu, 25 Aug 2011 14:34:44 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: Hans Petter Selasky , John Baldwin References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <201108251235.15853.hselasky@c2i.net> In-Reply-To: <201108251235.15853.hselasky@c2i.net> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 11:34:50 -0000 on 25/08/2011 13:35 Hans Petter Selasky said the following: > On Tuesday 23 August 2011 15:11:08 John Baldwin wrote: >>> I. [Why] do we need this pattern? >>> Can the code be re-written in a smarter (if not to say proper) way? >> > > Hi, > >> Giant is recursive, it should just be always acquired. Also, this >> recursive call pattern is very non-obvious. A far more straightforward >> approach would be to just have: > > Unless Witness is changed, that won't work. It will lead to LOR warnings I > think. > > Imagine that the root function locks Giant, then another lock is locked and > then ukbd_poll() is called. Won't the second lock of Giant cause a LOR > warning? I think no. At least that's how I interpret the following snippet in witness_checkorder(): /* * Check to see if we are recursing on a lock we already own. If * so, make sure that we don't mismatch exclusive and shared lock * acquires. */ lock1 = find_instance(lock_list, lock); if (lock1 != NULL) { if ((lock1->li_flags & LI_EXCLUSIVE) != 0 && (flags & LOP_EXCLUSIVE) == 0) { printf("shared lock of (%s) %s @ %s:%d\n", class->lc_name, lock->lo_name, file, line); printf("while exclusively locked from %s:%d\n", lock1->li_file, lock1->li_line); panic("share->excl"); } if ((lock1->li_flags & LI_EXCLUSIVE) == 0 && (flags & LOP_EXCLUSIVE) != 0) { printf("exclusive lock of (%s) %s @ %s:%d\n", class->lc_name, lock->lo_name, file, line); printf("while share locked from %s:%d\n", lock1->li_file, lock1->li_line); panic("excl->share"); } return; } Because of the return statement we do not seem to be doing any additional order checking in the case of recursion. >> static int >> ukbd_poll(keyboard_t *kbd, int on) >> { >> struct ukbd_softc *sc = kbd->kb_data; >> >> mtx_lock(&Giant); >> if (on) { >> sc->sc_flags |= UKBD_FLAG_POLLING; >> sc->sc_poll_thread = curthread; >> } else { >> sc->sc_flags &= ~UKBD_FLAG_POLLING; >> ukbd_start_timer(sc); /* start timer */ >> } >> mtx_unlock(&Giant); >> return (0); >> } >> > >> Most code should not be abusing mtx_owned() in this fashion. For Giant you >> should just follow a simple pattern like above that lets it recurse. For >> your own locks you generally should use things like mtx_assert() to >> require all callers of a given routine to hold the lock rather than >> recursively acquiring the lock. Very few legitimate cases of mtx_owned() >> exist IMO. It is debatable if we should even have a mtx_owned() routine >> since we have mtx_assert(). > > How would you solve the LOR case? > > --HPS -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 11:55:05 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16F7C106566B; Thu, 25 Aug 2011 11:55:05 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe06.c2i.net [212.247.154.162]) by mx1.freebsd.org (Postfix) with ESMTP id 67CEC8FC0C; Thu, 25 Aug 2011 11:55:04 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=RCcoR6dhO4r2plSBAg1vtKNWl/yGWcPYGWoK9vAZDcU= c=1 sm=1 a=SvYTsOw2Z4kA:10 a=ni9ZY7ZFbhgA:10 a=WQU8e4WWZSUA:10 a=8nJEP1OIZ-IA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=6I5d2MoRAAAA:8 a=N3vH5299AAAA:8 a=C4Pu0OO8ZFxwWlAcDLMA:9 a=wPNLvfGTeEIA:10 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe06.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 169843452; Thu, 25 Aug 2011 13:55:02 +0200 From: Hans Petter Selasky To: Andriy Gapon Date: Thu, 25 Aug 2011 13:52:31 +0200 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4E53986B.5000804@FreeBSD.org> <201108251308.56737.hselasky@c2i.net> <4E562E3A.7030304@FreeBSD.org> In-Reply-To: <4E562E3A.7030304@FreeBSD.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108251352.31504.hselasky@c2i.net> Cc: freebsd-arch@freebsd.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 11:55:05 -0000 > http://people.freebsd.org/~avg/stop_scheduler_on_panic.diff > http://people.freebsd.org/~avg/stop_scheduler_on_panic.8.x.diff The following patch complements Andriy's patch with regard to USB. Please test and report back! http://hselasky.homeunix.org:8192/usb_scheduler_stopped.patch MD5 (usb_scheduler_stopped.patch) = 45f019de180ca06982fc7df77e18d2fe --HPS From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 11:59:09 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 437DD1065673 for ; Thu, 25 Aug 2011 11:59:09 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id F2EFD8FC17 for ; Thu, 25 Aug 2011 11:59:08 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QwYaR-0000sR-Ur for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 13:59:07 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 13:59:07 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 13:59:07 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Thu, 25 Aug 2011 13:58:55 +0200 Lines: 16 Message-ID: References: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 In-Reply-To: X-Enigmail-Version: 1.1.2 Subject: Re: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 11:59:09 -0000 On 24/08/2011 18:18, Bjoern A. Zeeb wrote: > On Mar 19, 2011, at 6:37 AM, George Neville-Neil wrote: >> kern.ipc.maxsockbuf: 262144 >> net.inet.tcp.recvbuf_max: 262144 >> net.inet.tcp.sendbuf_max: 262144 >> >> I believe it's time to up these values to something that's in line with higher speed >> local networks, such as 10G. Perhaps it's time to move these to 2MB instead of 256K. >> >> Thoughts? > > > This never happened, did it? Was there a reason? While at it, did sshd buffer tuning happened? From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 12:00:09 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4261106566C; Thu, 25 Aug 2011 12:00:09 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5042D8FC29; Thu, 25 Aug 2011 12:00:09 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id E6F1B46B2D; Thu, 25 Aug 2011 08:00:08 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 88A3D8A02F; Thu, 25 Aug 2011 08:00:08 -0400 (EDT) From: John Baldwin To: Andriy Gapon Date: Thu, 25 Aug 2011 08:00:07 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <4E53986B.5000804@FreeBSD.org> <201108251235.15853.hselasky@c2i.net> <4E563354.5020106@FreeBSD.org> In-Reply-To: <4E563354.5020106@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108250800.07467.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Thu, 25 Aug 2011 08:00:08 -0400 (EDT) Cc: freebsd-arch@freebsd.org, Hans Petter Selasky Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 12:00:09 -0000 On Thursday, August 25, 2011 7:34:44 am Andriy Gapon wrote: > on 25/08/2011 13:35 Hans Petter Selasky said the following: > > On Tuesday 23 August 2011 15:11:08 John Baldwin wrote: > >>> I. [Why] do we need this pattern? > >>> Can the code be re-written in a smarter (if not to say proper) way? > >> > > > > Hi, > > > >> Giant is recursive, it should just be always acquired. Also, this > >> recursive call pattern is very non-obvious. A far more straightforward > >> approach would be to just have: > > > > Unless Witness is changed, that won't work. It will lead to LOR warnings I > > think. > > > > Imagine that the root function locks Giant, then another lock is locked and > > then ukbd_poll() is called. Won't the second lock of Giant cause a LOR > > warning? > > I think no. > At least that's how I interpret the following snippet in witness_checkorder(): > /* > * Check to see if we are recursing on a lock we already own. If > * so, make sure that we don't mismatch exclusive and shared lock > * acquires. > */ > lock1 = find_instance(lock_list, lock); > if (lock1 != NULL) { > if ((lock1->li_flags & LI_EXCLUSIVE) != 0 && > (flags & LOP_EXCLUSIVE) == 0) { > printf("shared lock of (%s) %s @ %s:%d\n", > class->lc_name, lock->lo_name, file, line); > printf("while exclusively locked from %s:%d\n", > lock1->li_file, lock1->li_line); > panic("share->excl"); > } > if ((lock1->li_flags & LI_EXCLUSIVE) == 0 && > (flags & LOP_EXCLUSIVE) != 0) { > printf("exclusive lock of (%s) %s @ %s:%d\n", > class->lc_name, lock->lo_name, file, line); > printf("while share locked from %s:%d\n", > lock1->li_file, lock1->li_line); > panic("excl->share"); > } > return; > } > > Because of the return statement we do not seem to be doing any additional order > checking in the case of recursion. Correct. WITNESS has never done LOR checking on trylocks or recursive acquires since those will never block. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 13:33:12 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 64E0D106564A; Thu, 25 Aug 2011 13:33:12 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 822558FC0C; Thu, 25 Aug 2011 13:33:11 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA05164; Thu, 25 Aug 2011 16:33:09 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E564F15.3010809@FreeBSD.org> Date: Thu, 25 Aug 2011 16:33:09 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: John Baldwin References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> In-Reply-To: <201108230911.09021.jhb@freebsd.org> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 13:33:12 -0000 on 23/08/2011 16:11 John Baldwin said the following: > On Tuesday, August 23, 2011 8:09:15 am Andriy Gapon wrote: >> >> Yes, the subject sounds quite hairy, so please let me try to explain it. >> First, let's consider one concrete function: >> >> static int >> ukbd_poll(keyboard_t *kbd, int on) >> { >> struct ukbd_softc *sc = kbd->kb_data; >> >> if (!mtx_owned(&Giant)) { >> /* XXX cludge */ >> int retval; >> mtx_lock(&Giant); >> retval = ukbd_poll(kbd, on); >> mtx_unlock(&Giant); >> return (retval); >> } >> >> if (on) { >> sc->sc_flags |= UKBD_FLAG_POLLING; >> sc->sc_poll_thread = curthread; >> } else { >> sc->sc_flags &= ~UKBD_FLAG_POLLING; >> ukbd_start_timer(sc); /* start timer */ >> } >> return (0); >> } >> >> This "XXX cludge" [sic] pattern is scattered around a few functions in the ukbd >> code and perhaps other usb code: >> func() >> { >> if (!mtx_owned(&Giant)) { >> mtx_lock(&Giant); >> func(); >> mtx_unlock(&Giant); >> return; >> } >> >> // etc ... >> } >> >> I have a few question directly and indirectly related to this pattern. >> >> I. [Why] do we need this pattern? >> Can the code be re-written in a smarter (if not to say proper) way? > > Giant is recursive, it should just be always acquired. Also, this recursive > call pattern is very non-obvious. A far more straightforward approach would > be to just have: > > static int > ukbd_poll(keyboard_t *kbd, int on) > { > struct ukbd_softc *sc = kbd->kb_data; > > mtx_lock(&Giant); > if (on) { > sc->sc_flags |= UKBD_FLAG_POLLING; > sc->sc_poll_thread = curthread; > } else { > sc->sc_flags &= ~UKBD_FLAG_POLLING; > ukbd_start_timer(sc); /* start timer */ > } > mtx_unlock(&Giant); > return (0); > } > Thank you for clarifying this, I agree this is simpler than the original code and should work exactly the same. There is more trouble with a few other functions that actually behave differently (even if slightly) depending on what mtx_owned(&Giant) returns. Not sure if that's a legal use or an antipattern. For example: ukbd_ioctl, ukbd_check, ukbd_check_char, ukbd_read, ukbd_read_char. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 13:45:47 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4572A106564A; Thu, 25 Aug 2011 13:45:47 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id F2A4B8FC16; Thu, 25 Aug 2011 13:45:46 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 9D4CB46B53; Thu, 25 Aug 2011 09:45:46 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id B482A8A037; Thu, 25 Aug 2011 09:45:45 -0400 (EDT) From: John Baldwin To: Andriy Gapon Date: Thu, 25 Aug 2011 09:45:24 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> In-Reply-To: <4E564F15.3010809@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108250945.24606.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Thu, 25 Aug 2011 09:45:46 -0400 (EDT) Cc: freebsd-arch@freebsd.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 13:45:47 -0000 On Thursday, August 25, 2011 9:33:09 am Andriy Gapon wrote: > on 23/08/2011 16:11 John Baldwin said the following: > > On Tuesday, August 23, 2011 8:09:15 am Andriy Gapon wrote: > >> > >> Yes, the subject sounds quite hairy, so please let me try to explain it. > >> First, let's consider one concrete function: > >> > >> static int > >> ukbd_poll(keyboard_t *kbd, int on) > >> { > >> struct ukbd_softc *sc = kbd->kb_data; > >> > >> if (!mtx_owned(&Giant)) { > >> /* XXX cludge */ > >> int retval; > >> mtx_lock(&Giant); > >> retval = ukbd_poll(kbd, on); > >> mtx_unlock(&Giant); > >> return (retval); > >> } > >> > >> if (on) { > >> sc->sc_flags |= UKBD_FLAG_POLLING; > >> sc->sc_poll_thread = curthread; > >> } else { > >> sc->sc_flags &= ~UKBD_FLAG_POLLING; > >> ukbd_start_timer(sc); /* start timer */ > >> } > >> return (0); > >> } > >> > >> This "XXX cludge" [sic] pattern is scattered around a few functions in the ukbd > >> code and perhaps other usb code: > >> func() > >> { > >> if (!mtx_owned(&Giant)) { > >> mtx_lock(&Giant); > >> func(); > >> mtx_unlock(&Giant); > >> return; > >> } > >> > >> // etc ... > >> } > >> > >> I have a few question directly and indirectly related to this pattern. > >> > >> I. [Why] do we need this pattern? > >> Can the code be re-written in a smarter (if not to say proper) way? > > > > Giant is recursive, it should just be always acquired. Also, this recursive > > call pattern is very non-obvious. A far more straightforward approach would > > be to just have: > > > > static int > > ukbd_poll(keyboard_t *kbd, int on) > > { > > struct ukbd_softc *sc = kbd->kb_data; > > > > mtx_lock(&Giant); > > if (on) { > > sc->sc_flags |= UKBD_FLAG_POLLING; > > sc->sc_poll_thread = curthread; > > } else { > > sc->sc_flags &= ~UKBD_FLAG_POLLING; > > ukbd_start_timer(sc); /* start timer */ > > } > > mtx_unlock(&Giant); > > return (0); > > } > > > > Thank you for clarifying this, I agree this is simpler than the original code and > should work exactly the same. > > There is more trouble with a few other functions that actually behave differently > (even if slightly) depending on what mtx_owned(&Giant) returns. Not sure if > that's a legal use or an antipattern. > > For example: ukbd_ioctl, ukbd_check, ukbd_check_char, ukbd_read, ukbd_read_char. I think many of these can be fixed the same way. The one issue I see so far is when things like ukbd_check() just fail if it is not polling and Giant is not held. You need to find out if that is due to similar misunderstandings about Giant LORs or not. If it is, you can probably just always acquire Giant. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 15:21:24 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E30F1065679 for ; Thu, 25 Aug 2011 15:21:24 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id DF1848FC1C for ; Thu, 25 Aug 2011 15:21:23 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Qwbk7-0003rA-R6 for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 17:21:19 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 17:21:19 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 17:21:19 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 25 Aug 2011 15:20:59 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 27 Message-ID: References: <705869186.20110819012421@serebryakov.spb.ru> <04EEADEE-380F-48A0-BBBF-1A1673228F90@cyberlifelabs.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Milo Hyson User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 15:21:24 -0000 Hi Milo Hyson! On Wed, 24 Aug 2011 19:42:23 -0700; Milo Hyson wrote about 'Re: FreeBSD problems and preliminary ways to solve': >>> walking all over the competition. Buzz is a critical part of selling ideas in >>> open source (for better or worse), and there's no reason we can't play in that >>> game a bit while maintaining our boring and staid personalities :-). >> >> Sure. And taking surveys into account, we could just simply summarize: >> FreeBSD needs marketing :-) > > That begs the question of to whom FreeBSD should be marketed. Home users? > Small-office admins? Datacenter admins? Embedded developers? Not quite. I think there are two questions, though related to each other. One is "who is FreeBSD target user". This influences "global" development decisions, such as roadmaps, what should sit in the base system, etc. The other is "which users to listen to and whom to advertise to", that is, marketing. And this doesn't need clear boundaries: e.g. we should listen to all our current users just to not lose the userbase. Later this will be ponetntial users, and so on. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 15:34:38 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D83B1065672 for ; Thu, 25 Aug 2011 15:34:38 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id EC2128FC0A for ; Thu, 25 Aug 2011 15:34:37 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Qwbwx-0002hV-Nr for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 17:34:35 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 17:34:35 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 17:34:35 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 25 Aug 2011 15:34:19 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 27 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: selven User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 15:34:38 -0000 Hi selven! On Tue, 23 Aug 2011 10:06:24 +0400; selven wrote about 'Re: FreeBSD problems and preliminary ways to solve': > Good post as well as pretty much scary. You mentionned about the purity of > FreeBSD at some point, that's one of the reasons why i prefer FreeBSD > (probably many of my friends who do work with it also), I've also said that purity doesn't always in conflict with newer world requirements. We can have both, it is solvable. > Since in a technical board, with 10 people, 7 > out of 10 are ubuntu users, with votes one always loses, and the funny thing > :s 7 out of 10 don't even know the difference between FreeBSD and linux, > they just voice out ubuntu because 'you will end up in trouble finding > people to maintain your stuffs if these 3 leaves' (weirdly we even fixes > there buntu bugs :s ). This user base is really a huge problem. The real problem here is that they're saying "trouble finding people". They are *right* here - there is too small number of FreeBSD'ers. That's our real problem - we should increase our userbase. Our userbase is the problem, not their's userbase. Until there's more of us, we can't fight successfully with them, even if our weapon is stronger. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 15:42:37 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3EF411065678 for ; Thu, 25 Aug 2011 15:42:37 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id EAE9E8FC38 for ; Thu, 25 Aug 2011 15:42:36 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Qwc4h-0007RZ-Vp for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 17:42:35 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 17:42:35 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 17:42:35 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 25 Aug 2011 15:42:20 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 30 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Garrett Cooper User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: being loud (Was: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 15:42:37 -0000 Hi Garrett Cooper! On Mon, 22 Aug 2011 23:54:35 -0700; Garrett Cooper wrote about 'Re: FreeBSD problems and preliminary ways to solve': >> And that's been my experience too. >> >> Linux users and advocates are very .. loud. FreeBSD users and >> advocates tend not to be so loud. >> Maybe that needs to change. > I also personally think that companies that use FreeBSD need to be > louder as well. There is Isilon, iXsystems, Juniper, etc, but there > are other companies that don't take a more active role in > acknowledging that they run FreeBSD under the hood which only > detriments FreeBSD's overall mindshare. > It's funny how certain companies like Dell, IBM, etc choose to be > so loud about their active Linux support -- so why should more > companies using FreeBSD also not take suit? So, you think. You think right. I agree, many others will agree. But what then? That's just words in arch@ - a little place for developers. Not much heared outside. Words, not actions. So, who will go and tell to that companies they should be loud?.. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 16:24:50 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A051106564A for ; Thu, 25 Aug 2011 16:24:50 +0000 (UTC) (envelope-from bf1783@googlemail.com) Received: from mail-ey0-f172.google.com (mail-ey0-f172.google.com [209.85.215.172]) by mx1.freebsd.org (Postfix) with ESMTP id 23DDD8FC0A for ; Thu, 25 Aug 2011 16:24:49 +0000 (UTC) Received: by eye4 with SMTP id 4so1601092eye.31 for ; Thu, 25 Aug 2011 09:24:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=x7VYpgLh0wyUcaXij2fGsRM7JxnLVqjrOnKG8BqEw5A=; b=Mb+ryDNHt+44G5yFfVL7udHQ0yvf8kn6kwJBXF3gn7Y20LqoD0BGgV7OFlMf00fdV3 lSFMZAkTx7HeZpepd5TjCuBimPkqOIVN+MpozoDnMyDzKGqM6J4BHh+CKUBmkxILD/Rz t+c/Sfv6nxSRbna1fsiRnB8Kk21X7C6VImXVs= MIME-Version: 1.0 Received: by 10.142.141.11 with SMTP id o11mr1829347wfd.56.1314287884464; Thu, 25 Aug 2011 08:58:04 -0700 (PDT) Received: by 10.68.66.106 with HTTP; Thu, 25 Aug 2011 08:58:04 -0700 (PDT) In-Reply-To: References: Date: Thu, 25 Aug 2011 15:58:04 +0000 Message-ID: From: "b. f." To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bf1783@gmail.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 16:24:50 -0000 On 8/25/11, b. f. wrote: >> >> I believe it's time to up these values to something that's in line with >> >> higher speed >> >> local networks, such as 10G. Perhaps it's time to move these to 2MB >> >> instead of 256K. >> >> >> >> Thoughts? >> > >> > >> > This never happened, did it? Was there a reason? >> > >> >> I went back and looked at the mail thread. I didn't see any strong >> objections >> so I think you should commit this for 9.x. >> >> np@ did point out that nmbclusters also lags on modern hardware so >> consider upping >> that at the same time. > > I thought Bruce's observation, in: > > http://lists.freebsd.org/pipermail/freebsd-arch/2011-March/011193.html > > that: > > "...there is an mostly-unrelated bufferbloat problem that is > purely local. If you have a buffer that is larger than an Ln cache (or > about half than), then actually using just a single buffer of that size > guarantees thrashing of the Ln cache, so that almost every memory access > is an Ln cache miss. Even with current hardware, a buffer of size 256K > will thrash most L1 caches and a buffer of size a few MB will thrash most > L2 caches." > > , and his suggestion for some sort of auto-tuning, deserve > consideration. Are you going to address this problem (at least the L2 > and higher cache thrashing), or give some suggestions for tuning in > UPDATING and the relevant manpages? Sorry, I should have sent this to -arch instead of -current. b. From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 16:53:36 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB0DE106566C for ; Thu, 25 Aug 2011 16:53:36 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 59DC58FC0C for ; Thu, 25 Aug 2011 16:53:35 +0000 (UTC) Received: from mart.js.berklix.net (p5DCBDCF8.dip.t-dialin.net [93.203.220.248]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p7PGZoDo049232; Thu, 25 Aug 2011 16:35:51 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id p7PGZcXo006354; Thu, 25 Aug 2011 18:35:39 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.4/8.14.4) with ESMTP id p7PGZQ2K054693; Thu, 25 Aug 2011 16:35:32 GMT (envelope-from jhs@fire.js.berklix.net) Message-Id: <201108251635.p7PGZQ2K054693@fire.js.berklix.net> To: vadim_nuclight@mail.ru From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Thu, 25 Aug 2011 15:42:20 -0000." Date: Thu, 25 Aug 2011 18:35:26 +0200 Sender: jhs@berklix.com Cc: freebsd-arch@freebsd.org Subject: Re: being loud (Was: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 16:53:37 -0000 > That's just words in arch@ - a little place for developers. Not much > heared outside. Words, not actions. > > So, who will go and tell to that companies they should be loud?.. There's a chance to be loud & advertise in 3 weeks time: http://lists.freebsd.org/pipermail/freebsd-advocacy/2011-August/004133.html But I'll omit detail here, to avoid distracting beyond list remit boundary. arch@ http://lists.freebsd.org/mailman/listinfo/freebsd-arch ... discussion of FreeBSD architecture. ... advocacy@ http://lists.freebsd.org/mailman/listinfo/freebsd-advocacy Furthering the Use of FreeBSD Share ideas and plan to increase the number of companies and individuals using FreeBSD Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Reply below, not above; Indent with "> "; Cumulative like a play script. Format: Plain text. Not HTML, multipart/alternative, base64, quoted-printable. From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 17:16:27 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 473CE106564A for ; Thu, 25 Aug 2011 17:16:27 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id AF2988FC08 for ; Thu, 25 Aug 2011 17:16:26 +0000 (UTC) Received: from mart.js.berklix.net (p57BCF11D.dip.t-dialin.net [87.188.241.29]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p7PHGOTv049488; Thu, 25 Aug 2011 17:16:25 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id p7PHGCp8006539; Thu, 25 Aug 2011 19:16:12 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.4/8.14.4) with ESMTP id p7PHFx2p055073; Thu, 25 Aug 2011 17:16:06 GMT (envelope-from jhs@fire.js.berklix.net) Message-Id: <201108251716.p7PHFx2p055073@fire.js.berklix.net> To: =?UTF-8?B?TWljaGFlbCBWLiBCdXp1dmVyb3Y=?= From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Fri, 19 Aug 2011 07:48:48 +0400." Date: Thu, 25 Aug 2011 19:15:59 +0200 Sender: jhs@berklix.com Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 17:16:27 -0000 Hi, > From: =?UTF-8?B?TWljaGFlbCBWLiBCdXp1dmVyb3Y=?= > Date: Fri, 19 Aug 2011 07:48:48 +0400 =?UTF-8?B?TWljaGFlbCBWLiBCdXp1dmVyb3Y=?= wrote: > --===============1194597977== > Content-Type: text/plain; charset=utf-8 > Content-Transfer-Encoding: base64 Please do not post Ascii text with Base 64 ! Though it works with http://lists.freebsd.org/pipermail/freebsd-arch/2011-August/011424.html It fails with http://docs.freebsd.org/cgi/getmsg.cgi?fetch=92494+0+archive/2011/freebsd-arch/20110821.freebsd-arch & fails with exmh-2.7.2 FreeBSD-8.2-RELEASE /usr/ports/mail/exmh2 & ptobably fails with search engines later. I had to mouse copy:... > Now, package creation process is "build -> install -> package". I believe that sequence "build -> package -> install" > is more correct and efficient. Yes, IMO that would sound an attractive capability. I guess there'd be issues to solve though :-) Maybe the attraction to ports architects not great though, as ports on the cluster get built with individual clean chroots I recall. So they may not see as high a percentage of breakage as end user systems (like eg inc. mine) that struggle to build the many local used ports not using chroots. I've long meant to find & read chroot scripts to build ports. ... Cos basic /usr/ports has never been coherently error free for me ( cd /var/db/pkg ; ls | wc -l # 1018 ) Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Reply below, not above; Indent with "> "; Cumulative like a play script. Format: Plain text. Not HTML, multipart/alternative, base64, quoted-printable. From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 18:16:02 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 290AB106564A for ; Thu, 25 Aug 2011 18:16:02 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id D50428FC12 for ; Thu, 25 Aug 2011 18:16:01 +0000 (UTC) Received: by qwc9 with SMTP id 9so1987817qwc.13 for ; Thu, 25 Aug 2011 11:16:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=M6c10ul+LR65am+MjXBHJF5F7ePn+ISZhpUZMdr8pb0=; b=OqNDKhrC61IrQBKuySG5xB1EoNCxC3TYxDUpW1dDKMr4F5p0gxx65dDf9htYBjvii8 SLeIngfF0iC51u4w5ljZhrwTmv+83Uds4sJKUlYWUNgBGmc5VBKKyO8QB8dsR9Dy05HH ywaIqZb6aEvaFDmXUod0v9ReF3E0SRCmDm3n0= MIME-Version: 1.0 Received: by 10.229.89.66 with SMTP id d2mr97273qcm.93.1314296161200; Thu, 25 Aug 2011 11:16:01 -0700 (PDT) Received: by 10.224.19.131 with HTTP; Thu, 25 Aug 2011 11:16:01 -0700 (PDT) In-Reply-To: <201108251716.p7PHFx2p055073@fire.js.berklix.net> References: <201108251716.p7PHFx2p055073@fire.js.berklix.net> Date: Thu, 25 Aug 2011 11:16:01 -0700 Message-ID: From: Garrett Cooper To: "Julian H. Stacey" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "Michael V. Buzuverov" , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 18:16:02 -0000 On Thu, Aug 25, 2011 at 10:15 AM, Julian H. Stacey wrote: > Hi, >> From: =A0 =A0 =A0 =A0 =3D?UTF-8?B?TWljaGFlbCBWLiBCdXp1dmVyb3Y=3D?=3D >> Date: =A0 =A0 =A0 =A0 Fri, 19 Aug 2011 07:48:48 +0400 > > =3D?UTF-8?B?TWljaGFlbCBWLiBCdXp1dmVyb3Y=3D?=3D wrote: >> --=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D1194597977=3D=3D >> Content-Type: text/plain; charset=3Dutf-8 >> Content-Transfer-Encoding: base64 > > Please do not post Ascii text with Base 64 ! > Though it works with > =A0http://lists.freebsd.org/pipermail/freebsd-arch/2011-August/011424.htm= l > It fails with > =A0http://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D92494+0+archive/2011/fr= eebsd-arch/20110821.freebsd-arch > & fails with exmh-2.7.2 > =A0 =A0 =A0 =A0FreeBSD-8.2-RELEASE /usr/ports/mail/exmh2 > & ptobably fails with search engines later. > > I had to mouse copy:... > >> Now, package creation process is "build -> install -> package". I believ= e that sequence "build -> package -> install" >> is more correct and efficient. The problem is that pkg_install wasn't properly designed to deal with chrooted package environments, and instead everything is installed directly to the target system and packaged from the target system (chroot functionality is really broken in pkg_add // pkg_create provided the right inputs, despite it being documented in the manpage :)..). If you do build -> package -> install, people that use ports will grumble and moan about the additional overhead required copying and installing things. But we really need to get out of the building everything from scratch business and into binary package installation, so yes.. I think the above proposed workflow makes sense. Even though some might complain that builds are now taking longer, it adds a minimal amount of overhead to do the above flow and is just cleaner and saner than the backwards way we currently do things in ports. And a lot of this can be fixed externally if people add a minimum amount of intelligence to their build systems to fine tune their build dependencies, instead of hammer approaches like I've seen in some build systems where things are nuked and rebuilt from scratch every time the build is run. Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 20:11:53 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5442D106566B for ; Thu, 25 Aug 2011 20:11:53 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0A1958FC17 for ; Thu, 25 Aug 2011 20:11:52 +0000 (UTC) Received: by ywo32 with SMTP id 32so2457522ywo.13 for ; Thu, 25 Aug 2011 13:11:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+lri86YdNY4YLj/NhAgMW5U6uxJjzrhdvB+SyX967t4=; b=Jt80yJ6VXiubvIfeQjXk9+PFg8nYC431SuJ5v1TOhh2gmETKPdkUCFR2imznGUy7oK XW4/GQwYY+mY3tfq6hKaKabX8k7VPeIGu5JyCw6AWuBtrN1RXPyYqGB+p4to5R6XzwAM 9qzMttEILcuwZ/3pQKma3x7sJEbZEpEdLxPbI= MIME-Version: 1.0 Received: by 10.150.115.7 with SMTP id n7mr1347412ybc.221.1314296443411; Thu, 25 Aug 2011 11:20:43 -0700 (PDT) Received: by 10.150.136.11 with HTTP; Thu, 25 Aug 2011 11:20:42 -0700 (PDT) In-Reply-To: References: <1313785806.56747.YahooMailClassic@web113503.mail.gq1.yahoo.com> <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> Date: Thu, 25 Aug 2011 11:20:42 -0700 Message-ID: From: Xin LI To: vadim_nuclight@mail.ru Content-Type: text/plain; charset=UTF-8 Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 20:11:53 -0000 Hi, Vadim, On Wed, Aug 24, 2011 at 3:30 PM, Vadim Goncharov wrote: > Hi Rick Macklem! > > On Fri, 19 Aug 2011 18:38:35 -0400 (EDT); Rick Macklem wrote about 'Re: FreeBSD problems and preliminary ways to solve': > >> One thing I thought I'd bring up (since I haven't seen it >> mentioned yet) is Debian GNU/kFreeBSD. I haven't tried it, >> so I'm talking through my hat a bunch, but... > >> It seems to me that FreeBSD should do what it can to support >> this effort. Why? Well, I suspect a lot of why organizations > > No. FreeBSD has limited resources. We need to: > > a) fix our own problems > b) develop our own unique features (see my message to Robert Watson) > > And supporting Debian will mean wasting resources which could be spent > to these. This will effectively kill FreeBSD as a separate entity if > our problems will stay unfixed and get worse due to this. > > Leave this work to Debian: they already have a wide community and many > resources. Any Linux distro has more resources than BSD because they just > only pack someone's software, and we have to also actually *develop* those > software (most of all, kernel and libc). I consider Debian GNU/kFreeBSD as a platform where we can see what the actual {performance,functionality} difference between our code and GNU ones, which is valuable. Supporting their effort might be time consuming and maybe not our priority, but it's good not to make their work harder because more importantly, their existence gives us more exposure and more eyes on our codebase as well. Cheers, -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 20:13:10 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 681801065673; Thu, 25 Aug 2011 20:13:10 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id 4E0FE8FC1E; Thu, 25 Aug 2011 20:13:10 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id 46CD02283B; Thu, 25 Aug 2011 14:13:04 -0600 (MDT) Received: by night.db.net (Postfix, from userid 1000) id A54465EC8; Thu, 25 Aug 2011 16:13:08 -0400 (EDT) Date: Thu, 25 Aug 2011 16:13:08 -0400 From: Diane Bruce To: Adrian Chadd Message-ID: <20110825201308.GA19658@night.db.net> References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: gnuyoga@gmail.com, Milo Hyson , owner-freebsd-arch@freebsd.org, freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 20:13:10 -0000 On Thu, Aug 25, 2011 at 01:24:54PM +0800, Adrian Chadd wrote: > On 25 August 2011 11:34, wrote: > >> > >> Sure. And taking surveys into account, we could just simply summarize: > >> FreeBSD needs marketing :-) > > > > That begs the question of to whom FreeBSD should be marketed. Home users? Small-office admins? Datacenter admins? Embedded developers? ... > Pick your area of interest. Work on making it more useful. Be very > loud and vocal about what you're doing. Explain how it's better than > the alternatives. Yes! > What _I_ think the project needs is louder developers and advocates; Yes. > So, let's stop talking about it; pick something you think should be > better, run with it, and be very vocal about what you do. I have been. ;-) Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db Why leave money to our children if we don't leave them the Earth? From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 20:29:51 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 546A11065670 for ; Thu, 25 Aug 2011 20:29:51 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id D6C308FC12 for ; Thu, 25 Aug 2011 20:29:50 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QwgYf-0004nZ-DU for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 22:29:49 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 22:29:49 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 22:29:49 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 25 Aug 2011 20:29:37 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 38 Message-ID: References: <1313785806.56747.YahooMailClassic@web113503.mail.gq1.yahoo.com> <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Xin LI User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 20:29:51 -0000 Hi Xin LI! On Thu, 25 Aug 2011 11:20:42 -0700; Xin LI wrote about 'Re: FreeBSD problems and preliminary ways to solve': >>> One thing I thought I'd bring up (since I haven't seen it >>> mentioned yet) is Debian GNU/kFreeBSD. I haven't tried it, >>> so I'm talking through my hat a bunch, but... >> >>> It seems to me that FreeBSD should do what it can to support >>> this effort. Why? Well, I suspect a lot of why organizations >> >> No. FreeBSD has limited resources. We need to: >> >> a) fix our own problems >> b) develop our own unique features (see my message to Robert Watson) >> >> And supporting Debian will mean wasting resources which could be spent >> to these. This will effectively kill FreeBSD as a separate entity if >> our problems will stay unfixed and get worse due to this. >> >> Leave this work to Debian: they already have a wide community and many >> resources. Any Linux distro has more resources than BSD because they just >> only pack someone's software, and we have to also actually *develop* those >> software (most of all, kernel and libc). > I consider Debian GNU/kFreeBSD as a platform where we can see what the > actual {performance,functionality} difference between our code and GNU > ones, which is valuable. Supporting their effort might be time > consuming and maybe not our priority, but it's good not to make their > work harder because more importantly, their existence gives us more > exposure and more eyes on our codebase as well. I agree that it's good not to make their work harder (you've pointed valid reasons), but I do not agree that their support is a priority to FreeBSD ("should do what it can"). That's auxiliary, not primary. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 20:52:57 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25809106567B for ; Thu, 25 Aug 2011 20:52:57 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id A895B8FC17 for ; Thu, 25 Aug 2011 20:52:56 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Qwgv1-00085C-Ip for freebsd-arch@freebsd.org; Thu, 25 Aug 2011 22:52:55 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 22:52:55 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 25 Aug 2011 22:52:55 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 25 Aug 2011 20:52:41 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 52 Message-ID: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Adrian Chadd User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 20:52:57 -0000 Hi Adrian Chadd! On Thu, 25 Aug 2011 13:24:54 +0800; Adrian Chadd wrote about 'Re: FreeBSD problems and preliminary ways to solve': >>> Sure. And taking surveys into account, we could just simply summarize: >>> FreeBSD needs marketing :-) >> >> That begs the question of to whom FreeBSD should be marketed. Home users? Small-office admins? Datacenter admins? Embedded developers? >> >> Perhaps we can start from what is currently used/deployed. Easier to start from what we have done than figuring out what all it can do. What say ? > Pick your area of interest. Work on making it more useful. Be very > loud and vocal about what you're doing. Explain how it's better than > the alternatives. > What _I_ think the project needs is louder developers and advocates; > easier install/management tools (especially for VM/cluster management > ; ports is a pain in the ass as viewed by a lot of people - who think > RPM/DEB is fine (and have built large networks around such)) and some > more use cases where FreeBSD makes sense. > So, let's stop talking about it; pick something you think should be > better, run with it, and be very vocal about what you do. Glad to hear such words from @FreeBSD.org! Thanks! > The project as a whole may not necessarily need a "project dictator" > per se. It doesn't need to figure out who FreeBSD needs to be > marketed to. Here an interesting question arise, in the philosophy/VCS field. We see that Linux/git adopted model where "dictator" has, say, 17 lieutenant for key subsystems, and pulls changes from them, each of them have, say, 17 own subordinates from whose he pulls, and so on. Instead of that 17^2 people FreeBSD has the same 289 men directly commiting to repository. It is repository here which acts as a "dictator" from technical side, and that is definetely better (e.g. no "kill -SIGBUS Linus" factors). The difference is, those 289 key people in Linux *can* pull changes from lower tiers, but FreeBSD people - can't (of course not at all, but it is significantly harder to contribute here). It is a plain model. So then, as they are using DVCS in a semi-centralized way, there are mostly technical benefits for them, which we don't have with a purely centralized old-fashioned SVN. I still think that it may be worth creating a combined "centralized DVCS" for FreeBSD needs, mirroring FreeBSD workflow and free from git flaws (listed in wiki, such as need to split to many repos etc.). Given how git initially was created, this is not so hard as it could sound. The problem here is, that I don't know what typical workflow a FreeBSD commiter has, and what exact needs are. So even as idea it is still blurred... -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 20:59:32 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 143BC106564A for ; Thu, 25 Aug 2011 20:59:32 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id C09B78FC0A for ; Thu, 25 Aug 2011 20:59:31 +0000 (UTC) Received: by qwc9 with SMTP id 9so2153644qwc.13 for ; Thu, 25 Aug 2011 13:59:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=hPmk3r5nUdBEz3c4Yu1PDGRoJJm9HX7bfDlT3+xsPYk=; b=n0RRuSCuDjLNgBA2my3gpsV9qKqYkaBDntA0xwKViuHSEe4J3QdDWNtl7dO2eXUNBb 22cBjjq0kJacHAphE4WM/q3PfPM77VXeQPuGSGmFPsxAiTMJI8LuXys5OL8G4bXtxCda qiIvEJknLY1hyRysLu1Ndrl+pmgmb9CtzNIgE= MIME-Version: 1.0 Received: by 10.229.87.137 with SMTP id w9mr266778qcl.284.1314305970917; Thu, 25 Aug 2011 13:59:30 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.98.8 with HTTP; Thu, 25 Aug 2011 13:59:30 -0700 (PDT) In-Reply-To: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Date: Thu, 25 Aug 2011 13:59:30 -0700 X-Google-Sender-Auth: LAYJ3c8ZrA4lemgjgZDn5bvqfyo Message-ID: From: mdf@FreeBSD.org To: vadim_nuclight@mail.ru Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 20:59:32 -0000 On Thu, Aug 25, 2011 at 1:52 PM, Vadim Goncharov wrote: > Here an interesting question arise, in the philosophy/VCS field. We see > that Linux/git adopted model where "dictator" has, say, 17 lieutenant > for key subsystems, and pulls changes from them, each of them have, say, > 17 own subordinates from whose he pulls, and so on. Instead of that 17^2 > people FreeBSD has the same 289 men directly commiting to repository. > It is repository here which acts as a "dictator" from technical side, > and that is definetely better (e.g. no "kill -SIGBUS Linus" factors). > The difference is, those 289 key people in Linux *can* pull changes from > lower tiers, but FreeBSD people - can't (of course not at all, but it is > significantly harder to contribute here). It is a plain model. I like that the Project is small enough that (1) I can be trusted to commit to any of it, and (2) after a few more years of work on it, I may very well know more than half of the code. It's not always possible to have lots of functionality in a small code base, but less code is better, and I wonder if FreeBSD's code size stays smaller because we can all work on all of it. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Thu Aug 25 22:09:29 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15CBB106564A for ; Thu, 25 Aug 2011 22:09:29 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 95AA88FC08 for ; Thu, 25 Aug 2011 22:09:27 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Qwi74-00006Q-DQ for freebsd-arch@freebsd.org; Fri, 26 Aug 2011 00:09:26 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 26 Aug 2011 00:09:26 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 26 Aug 2011 00:09:26 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 25 Aug 2011 22:09:07 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 55 Message-ID: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: mdf@FreeBSD.org User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: VCS (Was: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2011 22:09:29 -0000 Hi mdf@FreeBSD.org! On Thu, 25 Aug 2011 13:59:30 -0700; mdf@FreeBSD.org wrote about 'Re: FreeBSD problems and preliminary ways to solve': >> Here an interesting question arise, in the philosophy/VCS field. We see >> that Linux/git adopted model where "dictator" has, say, 17 lieutenant >> for key subsystems, and pulls changes from them, each of them have, say, >> 17 own subordinates from whose he pulls, and so on. Instead of that 17^2 >> people FreeBSD has the same 289 men directly commiting to repository. >> It is repository here which acts as a "dictator" from technical side, >> and that is definetely better (e.g. no "kill -SIGBUS Linus" factors). >> The difference is, those 289 key people in Linux *can* pull changes from >> lower tiers, but FreeBSD people - can't (of course not at all, but it is >> significantly harder to contribute here). It is a plain model. > I like that the Project is small enough that (1) I can be trusted to > commit to any of it, and (2) after a few more years of work on it, I > may very well know more than half of the code. It's not always > possible to have lots of functionality in a small code base, Umm, may be I was not clear. The question of trust is orthogonal to what I wanted to say. If the VCS machine is a "dictator", then any committer is trusted to commit to any part of repo, this is left as it is now. What I'm talking about is contributing. I can view a part of committer's workflow when a contributor wants to do something big. So, he has to do things in his own repo first, then make a big patch to send to his friend at freebsd.org, which will apply it and commit. Many boilerplate work. But there will be much more work if contributor periodically updates his work. It is will be a pain to merge changes. And with DVCS all history in contributor's repo could been merged to main FreeBSD repo as well. Think about Perforce and users/ & projects/ in our SVN - that's also too much overhead for most contributors. Suppose we have such VCS based around central repo. Because a central repo concept exists, I, as a contributor, do clone a *set of files*, neither just subtree as in SVN nor entire repository as in Git. The VCS maintains those set - sys/netinet/ipfw/*, sbin/ipfw/*, etc/rc.d/ipfw and a few files from sys/netinet/ (not all). Note those are individual files, based on a inode-like object in repo. My VCS copy set up to templates as in main tree (just like subversion-freebsd now). I do several commits to those files, the VCS automatically tracks changes from central repo to me. Then, when I'm done, those entire commit history from my branch could be imported by interested committer, not just resulting patch. Effectively an outside vendor branch, closely tied to it's native original freebsd.org, though. > but less code is better, and I wonder if FreeBSD's code size stays > smaller because we can all work on all of it. Nope. More solved tasks - that is what better. And less code/smaller & cleaner code/less people/etc. - these are just a manner, a way to achieve that goal, not the goal itself. Adjective is not a noun. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 00:35:52 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 33201106564A for ; Fri, 26 Aug 2011 00:35:52 +0000 (UTC) (envelope-from jamesbrandongooch@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id B62968FC0A for ; Fri, 26 Aug 2011 00:35:51 +0000 (UTC) Received: by wyh15 with SMTP id 15so2601405wyh.13 for ; Thu, 25 Aug 2011 17:35:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=+IHPlqlZf4ZxXx4EZ2FRFldNEe24xCZpHsIaK2UebIo=; b=PrKac++BqRu0IcvX3pLXa6fSOk5cT+KALFCYDBfZY4r8E3IwVwmt7DeygMFiONBi0X TKaihfgR2IhqEjFaNZU0ZVq3qa42c1pt/zW0nBSOt+9RPVHkzSW5/2o7zVTp9vPE7gnq g9jIbBSTGyZtnCFS7nMKl/y7sA48wS87Ufe5M= MIME-Version: 1.0 Received: by 10.216.88.212 with SMTP id a62mr316700wef.43.1314317558008; Thu, 25 Aug 2011 17:12:38 -0700 (PDT) Received: by 10.216.208.158 with HTTP; Thu, 25 Aug 2011 17:12:37 -0700 (PDT) In-Reply-To: <04EEADEE-380F-48A0-BBBF-1A1673228F90@cyberlifelabs.com> References: <705869186.20110819012421@serebryakov.spb.ru> <04EEADEE-380F-48A0-BBBF-1A1673228F90@cyberlifelabs.com> Date: Thu, 25 Aug 2011 19:12:37 -0500 Message-ID: From: Brandon Gooch To: Milo Hyson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 00:35:52 -0000 On Wed, Aug 24, 2011 at 9:42 PM, Milo Hyson wrote: > On Aug 24, 2011, at 3:09 PM, Vadim Goncharov wrote: > >>> walking all over the competition. =A0Buzz is a critical part of selling= ideas in >>> open source (for better or worse), and there's no reason we can't play = in that >>> game a bit while maintaining our boring and staid personalities :-). >> >> Sure. And taking surveys into account, we could just simply summarize: >> FreeBSD needs marketing :-) > > That begs the question of to whom FreeBSD should be marketed. Home users?= Small-office admins? Datacenter admins? Embedded developers? > > - Milo Hyson > Chief Scientist > CyberLife Labs, Inc. > FreeBSD should be marketed to DEVELOPERS. Users of all skill levels have needs, wants, and ideas. Developers are the ones who implement these things in code. I think the question is "how do we lure the developers?". FreeBSD is well documented for an open source project. In particular, the Handbook serves as an excellent guide and reference for FreeBSD from an end-user's perspective. But is the documentation for developers as well-structured? I'd like to hear stories from the devs out there in this regard. Perhaps the FreeBSD current developer community (see: decades of experience and knowledge) should focus on the creation (or revision) of solid, comprehensive documentation for developing software in the FreeBSD environment. Even something as simple as this forum post is an awesome place to start: http://forums.freebsd.org/showthread.php?t=3D1566 Maybe a restructuring of the primary FreeBSD website is in order, with an emphasis on marketing to developers, from the fresh (university students) to the seasoned (industry minds involved in company decision-making process). I believe that a beneficial side-effect of being a developer-centric OS will the eventual refinement of FreeBSD as a "first choice" desktop operating system. Seems to me that developers are more likely to use the operating system for day-to-day "desktop" tasks, fixing and adding features they require (eventually moving away from running Mac OS X or some Linux distro on their laptops to running FreeBSD primarily). Eventually, enough development occurs in key areas such as hardware support and features (reliable suspend/resume, Wireless N, KMS/GEM, etc...), that it's feasible to imagine a group spinning their own "distro" -- would that really be so bad? We don't seem to mind the PC-BSD folks, who are doing a fine job as things stand :) Imagine a horde of new college graduates, with FreeBSD under their belts (instead of some Linux distribution), ready to deploy it as soon as they have the chance in their new roles as system administrators and engineers -- sounds great to me. More bodies, more eyes, more minds -- this brings along with it more energy. We should focus on making FreeBSD the most developer-friendly OS out there. -Brandon From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 02:13:04 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE32A1065677; Fri, 26 Aug 2011 02:13:04 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (unknown [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id BBB3B8FC19; Fri, 26 Aug 2011 02:13:04 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id p7Q2D39t003876 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 25 Aug 2011 19:13:04 -0700 (PDT) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id p7Q2D3gq003875; Thu, 25 Aug 2011 19:13:03 -0700 (PDT) Received: from fbsd81 ([192.168.200.81]) by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA01980; Thu, 25 Aug 11 19:03:01 PDT Date: Fri, 26 Aug 2011 02:02:31 -0700 From: perryh@pluto.rain.com To: jhb@freebsd.org Message-Id: <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> In-Reply-To: <201108250945.24606.jhb@freebsd.org> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: avg@freebsd.org, freebsd-arch@freebsd.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 02:13:05 -0000 John Baldwin wrote: > ... acquire Giant. Last I heard, Giant was dropped in 8.0 (and that was the reason for isdn and sio being dropped). Has it been reinstated? If so, should isdn and sio also be reinstated? From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 06:36:55 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B335A1065670; Fri, 26 Aug 2011 06:36:55 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B86BC8FC15; Fri, 26 Aug 2011 06:36:54 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA18004; Fri, 26 Aug 2011 09:36:46 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Qwq21-000HAA-Q9; Fri, 26 Aug 2011 09:36:45 +0300 Message-ID: <4E573EFB.7000800@FreeBSD.org> Date: Fri, 26 Aug 2011 09:36:43 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0) Gecko/20110819 Thunderbird/6.0 MIME-Version: 1.0 To: perryh@pluto.rain.com References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> In-Reply-To: <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: jhb@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 06:36:55 -0000 on 26/08/2011 12:02 perryh@pluto.rain.com said the following: > Last I heard, Giant was dropped in 8.0 (and that was the reason for > isdn and sio being dropped). You heard incorrectly or misunderstood what you heard. Just search the sources for Giant. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 07:00:38 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C686106566B; Fri, 26 Aug 2011 07:00:38 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe03.c2i.net [212.247.154.66]) by mx1.freebsd.org (Postfix) with ESMTP id 34A118FC17; Fri, 26 Aug 2011 07:00:36 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=yVKV3zusvCapyMfYJBNW2j35FMEuTKq6vh/tt/1L5+g= c=1 sm=1 a=SvYTsOw2Z4kA:10 a=ni9ZY7ZFbhgA:10 a=WQU8e4WWZSUA:10 a=8nJEP1OIZ-IA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=GKysJfYJAAAA:8 a=LflPMe0w_8KeAaEYuW4A:9 a=wPNLvfGTeEIA:10 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe03.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 3337180; Fri, 26 Aug 2011 09:00:34 +0200 From: Hans Petter Selasky To: freebsd-arch@freebsd.org Date: Fri, 26 Aug 2011 08:58:02 +0200 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4E53986B.5000804@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> In-Reply-To: <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108260858.02411.hselasky@c2i.net> Cc: perryh@pluto.rain.com, avg@freebsd.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 07:00:38 -0000 On Friday 26 August 2011 11:02:31 perryh@pluto.rain.com wrote: > isdn Regarding ISDN, there is a Giant-free ISDN stack for FreeBSD out of the tree. It probably needs a little style work before committed, though technically it is very Ok. --HPS From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 07:05:02 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 374C9106566B; Fri, 26 Aug 2011 07:05:02 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe07.c2i.net [212.247.154.194]) by mx1.freebsd.org (Postfix) with ESMTP id 859588FC12; Fri, 26 Aug 2011 07:05:01 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=We2KpSpXIp7zua8olfHtbA6oPL2p8ijoExYxXUNIRvU= c=1 sm=1 a=SvYTsOw2Z4kA:10 a=ni9ZY7ZFbhgA:10 a=WQU8e4WWZSUA:10 a=8nJEP1OIZ-IA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=GKysJfYJAAAA:8 a=oyWlK0wCT5cHk6VARVoA:9 a=wPNLvfGTeEIA:10 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe07.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 170849742; Fri, 26 Aug 2011 09:04:47 +0200 From: Hans Petter Selasky To: freebsd-arch@freebsd.org Date: Fri, 26 Aug 2011 09:02:15 +0200 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4E53986B.5000804@FreeBSD.org> <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> <201108260858.02411.hselasky@c2i.net> In-Reply-To: <201108260858.02411.hselasky@c2i.net> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108260902.15554.hselasky@c2i.net> Cc: perryh@pluto.rain.com, avg@freebsd.org Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 07:05:02 -0000 On Friday 26 August 2011 08:58:02 Hans Petter Selasky wrote: > On Friday 26 August 2011 11:02:31 perryh@pluto.rain.com wrote: > > isdn > > Regarding ISDN, there is a Giant-free ISDN stack for FreeBSD out of the > tree. It probably needs a little style work before committed, though > technically it is very Ok. Probably this stuff will end up like a port. --HPS From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 08:59:12 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72E51106564A for ; Fri, 26 Aug 2011 08:59:12 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 2C57C8FC15 for ; Fri, 26 Aug 2011 08:59:12 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 52FDA46B0D; Fri, 26 Aug 2011 04:59:11 -0400 (EDT) Date: Fri, 26 Aug 2011 09:59:11 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Xin LI In-Reply-To: Message-ID: References: <1313785806.56747.YahooMailClassic@web113503.mail.gq1.yahoo.com> <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: vadim_nuclight@mail.ru, freebsd-arch@freebsd.org Subject: Debian/kFreeBSD (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 08:59:12 -0000 On Thu, 25 Aug 2011, Xin LI wrote: >>> It seems to me that FreeBSD should do what it can to support this effort. >>> Why? Well, I suspect a lot of why organizations >> >> No. FreeBSD has limited resources. We need to: >> >> a) fix our own problems >> b) develop our own unique features (see my message to Robert Watson) >> >> And supporting Debian will mean wasting resources which could be spent to >> these. This will effectively kill FreeBSD as a separate entity if our >> problems will stay unfixed and get worse due to this. >> >> Leave this work to Debian: they already have a wide community and many >> resources. Any Linux distro has more resources than BSD because they just >> only pack someone's software, and we have to also actually *develop* those >> software (most of all, kernel and libc). > > I consider Debian GNU/kFreeBSD as a platform where we can see what the > actual {performance,functionality} difference between our code and GNU ones, > which is valuable. Supporting their effort might be time consuming and > maybe not our priority, but it's good not to make their work harder because > more importantly, their existence gives us more exposure and more eyes on > our codebase as well. Debian GNU/kFreeBSD offers a number of interesting prospects: (1) Side-by-side comparison between FreeBSD and Linux kernels + features using a mature and complete userspace (2) Allow Debian users easy access to features hard to get with a stock Linux kernel: pf, 802.11 stack, Netgraph, DTrace, ZFS, Capsicum, etc. (3) Allow easier hosting of mature Linux environments on top of hybrid FreeBSD/Linux hosting: provide Linux jails on FreeBSD ISPs but with the many benefits of using the native ABI. (4) Potential stepping stone on the path from Linux to FreeBSD (or, to be fair, vice versa!). (5) Increase exposure of FreeBSD kernel features and approaches in the broder OS community. (6) Provide another motivation for third-party application developers to consider FreeBSD-specific kernel features viable for use. For example, if Debian/kFreeBSD makes developing Capsicum-aware applications on a Linux-like platform easy, then FreeBSD ports wins as well. (7) Provide an additional set of interested hands when it comes to improving the clarity of definition and robustness of our system call ABI. This is something we've been increasing our focus on over time, and especially if the Debian folk track our development HEAD, they can provide us much earlier feedback either when we mess up, or when there's something we can do to make things easier for them (and therefore, likely, us in the future). (8) Use of the Debian "brand" to promote FreeBSD as a vibrant and interesting thing for the broader community of Linux users. I think we should be supporting the Debian/kFreeBSD folks in their effort -- hence inviting them to join our developer summits, accepting patches, etc. As both the FreeBSD and the Debian communities pride themselves in being a bit persnickety about the spelling and details, there will inevitably be some disagreements about approach. However, we may find that we can make their lives a lot easier by making relatively small changes to our kernel, and that they have extremely useful feedback to bring us on our kernel implementation. Robert From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 09:06:33 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9D77106566B; Fri, 26 Aug 2011 09:06:33 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 83B718FC17; Fri, 26 Aug 2011 09:06:33 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 0A57F46B0D; Fri, 26 Aug 2011 05:06:33 -0400 (EDT) Date: Fri, 26 Aug 2011 10:06:32 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: mdf@FreeBSD.org In-Reply-To: Message-ID: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: vadim_nuclight@mail.ru, freebsd-arch@freebsd.org Subject: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 09:06:34 -0000 On Thu, 25 Aug 2011, mdf@FreeBSD.org wrote: > On Thu, Aug 25, 2011 at 1:52 PM, Vadim Goncharov > wrote: >> Here an interesting question arise, in the philosophy/VCS field. We see >> that Linux/git adopted model where "dictator" has, say, 17 lieutenant for >> key subsystems, and pulls changes from them, each of them have, say, 17 own >> subordinates from whose he pulls, and so on. Instead of that 17^2 people >> FreeBSD has the same 289 men directly commiting to repository. It is >> repository here which acts as a "dictator" from technical side, and that is >> definetely better (e.g. no "kill -SIGBUS Linus" factors). The difference >> is, those 289 key people in Linux *can* pull changes from lower tiers, but >> FreeBSD people - can't (of course not at all, but it is significantly >> harder to contribute here). It is a plain model. > > I like that the Project is small enough that (1) I can be trusted to commit > to any of it, and (2) after a few more years of work on it, I may very well > know more than half of the code. It's not always possible to have lots of > functionality in a small code base, but less code is better, and I wonder if > FreeBSD's code size stays smaller because we can all work on all of it. I'm less concerned about the code base size, and more concerned about how we can best support an ecosystem of FreeBSD-derived projects. FreeNAS, PC-BSD, pfSense, zrouter, etc, but also companies like Juniper, Yahoo!, NetApp, Panasas, EMC, etc, all want to do two things to FreeBSD: (1) Specialise aspects of the FreeBSD environment (2) Extend aspects of the FreeBSD environment Where possible, we should integrate changes that make their lives easier, of course (and do -- several PC-BSD developers were given FreeBSD commit bits specifically so that they could merge back PC-BSD changes). However, we do have to ask ourselves if our revision control model makes their lives easier. I personally believe that the model we currently use scales quite well (subject to some unfortunate limitations in Subversion merging support) for what we're trying to accomplish as the FreeBSD project. However, one aspect of the git/Mercurial/etc model that we aren't able to capture particularly well is making it easy for people to pull in FreeBSD as a "flow of changesets" to use as a foundation for their own projects. I've visited a number of companies tracking FreeBSD, and they all find this difficult -- not just the social model of how to do it, but also simply the technical obstacles. Many use Perforce and find that works well, but I think we need to provide stronger support for the downstream open source community. One way to do this is to make "more official" our output if git exports of the repository -- something that many other Subversion-based projects do: Chromium, clang/LLVM, Tor, etc. Folk like Ulrich have been doing this on a casual basis for some time, but I think we need to formalise this and provide end-user documentation on how to use git to track FreeBSD, contribute patches, etc. There are a number of hitches people have to know about: the potential impact of obliteration, how to handle $FreeBSD$ correctly for system call additions, and so on. Simply writing them down and having an official git.FreeBSD.org (or even gitsvn.FreeBSD.org) would go a long way. I have to admit I've always preferred Perforce to git, simply because it strikes me as a more structured approach, partial checkouts (but especially composition of different depot pieces in a single checkout to create hybrid trees), etc. But git is widely used, and quite effectively used, by large communities. We need to support those communities better. Robert From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 09:11:37 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 403A0106564A; Fri, 26 Aug 2011 09:11:37 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1723C8FC0C; Fri, 26 Aug 2011 09:11:37 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 9064C46B0D; Fri, 26 Aug 2011 05:11:36 -0400 (EDT) Date: Fri, 26 Aug 2011 10:11:36 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: perryh@pluto.rain.com In-Reply-To: <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> Message-ID: References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: avg@freebsd.org, freebsd-arch@freebsd.org Subject: Removing Giant from VFS in 10.0 (was: Re: skipping locks, mutex_owned, usb) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 09:11:37 -0000 On Fri, 26 Aug 2011, perryh@pluto.rain.com wrote: > John Baldwin wrote: > >> ... acquire Giant. > > Last I heard, Giant was dropped in 8.0 (and that was the reason for isdn and > sio being dropped). Has it been reinstated? If so, should isdn and sio > also be reinstated? Giant persists in a number of places in the kernel, although most mainstream subsystems have been Giant-free for several years. Giant compatibility was removed from the network stack for 8, and I had hoped Giant compatibility would be removed from VFS for 9. That hasn't happened, but there are signs it may happen for 10. The main constraints there are that a pool of useful file systems remains unconverted to the new locking world order -- particularly, smbfs. I'd love to see someone take the bull by the horns on this one (Attilio?) -- the process we used for the network stack can probably be replicated there without too much difficulty: (1) Enumerate all remaining file systems dependent on Giant (probably already in the wiki, but review and update). (2) Post a Giant deprecation plan for VFS to arch@, current@, and fs@, allowing for substantial windows of time between "Announce removal plans", "Disable build of non-conforming parts", "Remove non-conforming parts", and with plenty of solicitations for help in fixing non-conforming parts. (3) Put in some of the legwork to help fix critical things -- mostly netsmb/smbfs. I will commit to either (a) making Coda MPSAFE for 10.0 or (b) removing it from the tree myself if I or someone else doesn't do it in time. But we do need more hands if we want to bring some of the legacy file systems forward -- especially things like nwfs/netncp, smbfs/netsmb, hpfs, etc. We also need to start announcing this early in the 10.0 cycle so that third-party file system developers for FreeBSD -- especially anyone interested in things like OpenAFS and Fuse, can do appropriate updates there as well. Robert From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 09:16:41 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C542D106564A; Fri, 26 Aug 2011 09:16:41 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5872E8FC0A; Fri, 26 Aug 2011 09:16:41 +0000 (UTC) Received: by gxk28 with SMTP id 28so3141280gxk.13 for ; Fri, 26 Aug 2011 02:16:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=y40zifDQWki808B+Xm+FBOyzAvuFJ8l8lOC78/EAtyc=; b=sdl9xJ8qenVzk64nkMH17T8c/01a7FR7U76kD9YQ/e0NkQcbCsjXg87edBbJQTACgX dNRsdOSIwKrTKT5lDkswHbi2obEL6R6Tu87ePa0D7p8gOX536YRsYLbv3ju11FSqvXB1 0A/WzILA1o3Grb1TasSjXXEGkN9QshYuXstk0= MIME-Version: 1.0 Received: by 10.150.32.14 with SMTP id f14mr1993639ybf.201.1314350200628; Fri, 26 Aug 2011 02:16:40 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.145.21 with HTTP; Fri, 26 Aug 2011 02:16:40 -0700 (PDT) In-Reply-To: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Date: Fri, 26 Aug 2011 17:16:40 +0800 X-Google-Sender-Auth: 9Cp-NY4fapfKSfGGeiL92UfqmX4 Message-ID: From: Adrian Chadd To: Robert Watson Content-Type: text/plain; charset=ISO-8859-1 Cc: vadim_nuclight@mail.ru, mdf@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 09:16:41 -0000 [snip] I've been trying to figure out how to actually _use_ git in a way that lets me do continuous (re)integration back from/to FreeBSD. Ie, being able to pull/rebase things from upstream, then push commits back into the tree, and then pull those back from upstream. There's git/SVN integration, but I've not seen examples of how it can be used by FreeBSD developers with SVN accounts; The examples of git/FreeBSD use that I've seen are for: * developers who don't have SVN accounts; * single projects that seem to stay separate from the main tree. Please help! Adrian From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 09:38:19 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4312E106566C for ; Fri, 26 Aug 2011 09:38:19 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 95E3C8FC08 for ; Fri, 26 Aug 2011 09:38:18 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Qwsrr-000PQ9-AQ; Fri, 26 Aug 2011 13:38:27 +0400 Date: Fri, 26 Aug 2011 13:38:27 +0400 From: Slawa Olhovchenkov To: Brandon Gooch Message-ID: <20110826093827.GA21676@zxy.spb.ru> References: <705869186.20110819012421@serebryakov.spb.ru> <04EEADEE-380F-48A0-BBBF-1A1673228F90@cyberlifelabs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: Milo Hyson , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 09:38:19 -0000 On Thu, Aug 25, 2011 at 07:12:37PM -0500, Brandon Gooch wrote: > On Wed, Aug 24, 2011 at 9:42 PM, Milo Hyson wrote: > > On Aug 24, 2011, at 3:09 PM, Vadim Goncharov wrote: > > > >>> walking all over the competition. šBuzz is a critical part of selling ideas in > >>> open source (for better or worse), and there's no reason we can't play in that > >>> game a bit while maintaining our boring and staid personalities :-). > >> > >> Sure. And taking surveys into account, we could just simply summarize: > >> FreeBSD needs marketing :-) > > > > That begs the question of to whom FreeBSD should be marketed. Home users? Small-office admins? Datacenter admins? Embedded developers? > > > > - Milo Hyson > > Chief Scientist > > CyberLife Labs, Inc. > > > > FreeBSD should be marketed to DEVELOPERS. > > Users of all skill levels have needs, wants, and ideas. Developers are > the ones who implement these things in code. I think the question is > "how do we lure the developers?". Needs for developers: VTune, OpenCL, CUDA, Java sertification platform, more stronge binary compatibility with older version. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 10:02:44 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50F331065673 for ; Fri, 26 Aug 2011 10:02:44 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id D33CD8FC08 for ; Fri, 26 Aug 2011 10:02:39 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QwtFF-00081m-6J for freebsd-arch@freebsd.org; Fri, 26 Aug 2011 12:02:37 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 26 Aug 2011 12:02:37 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 26 Aug 2011 12:02:37 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Fri, 26 Aug 2011 10:02:25 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 30 Message-ID: References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Robert Watson User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: Removing Giant from VFS in 10.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 10:02:44 -0000 Hi Robert Watson! On Fri, 26 Aug 2011 10:11:36 +0100 (BST); Robert Watson wrote about 'Removing Giant from VFS in 10.0 (was: Re: skipping locks, mutex_owned, usb)': > the process we used for the network stack can probably be replicated there > without too much difficulty: > (1) Enumerate all remaining file systems dependent on Giant (probably already > in the wiki, but review and update). > (2) Post a Giant deprecation plan for VFS to arch@, current@, and fs@, > allowing for substantial windows of time between "Announce removal plans", > "Disable build of non-conforming parts", "Remove non-conforming parts", > and with plenty of solicitations for help in fixing non-conforming parts. > (3) Put in some of the legwork to help fix critical things -- mostly > netsmb/smbfs. [...] > We also need to start announcing this early in the 10.0 cycle so that > third-party file system developers for FreeBSD -- especially anyone interested > in things like OpenAFS and Fuse, can do appropriate updates there as well. And, given the old and recent threads, to announce@ too, please. With, presumably, call for volunteers/sponsors. Or all that last week's hype was for nothing?.. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 10:29:10 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FF87106566C; Fri, 26 Aug 2011 10:29:10 +0000 (UTC) (envelope-from jra40@hermes.cam.ac.uk) Received: from ppsw-50.csi.cam.ac.uk (ppsw-50.csi.cam.ac.uk [131.111.8.150]) by mx1.freebsd.org (Postfix) with ESMTP id 22D668FC1A; Fri, 26 Aug 2011 10:29:09 +0000 (UTC) X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-qy0-f182.google.com ([209.85.216.182]:52678) by ppsw-50.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:465) with esmtpsa (PLAIN:jra40) (TLSv1:RC4-SHA:128) id 1QwtHA-0007qO-rt (Exim 4.72) (return-path ); Fri, 26 Aug 2011 11:04:36 +0100 Received: by qyk9 with SMTP id 9so2469977qyk.13 for ; Fri, 26 Aug 2011 03:04:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.43.203 with SMTP id x11mr1208637qce.127.1314353072600; Fri, 26 Aug 2011 03:04:32 -0700 (PDT) Received: by 10.229.232.196 with HTTP; Fri, 26 Aug 2011 03:04:32 -0700 (PDT) In-Reply-To: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Date: Fri, 26 Aug 2011 11:04:32 +0100 Message-ID: From: Jonathan Anderson To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 Sender: Jonathan Anderson Cc: vadim_nuclight@mail.ru, mdf@freebsd.org, Robert Watson , freebsd-arch@freebsd.org Subject: Re: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 10:29:10 -0000 On 26 August 2011 10:16, Adrian Chadd wrote: > [snip] > > I've been trying to figure out how to actually _use_ git in a way that > lets me do continuous (re)integration back from/to FreeBSD. > Ie, being able to pull/rebase things from upstream, then push commits > back into the tree, and then pull those back from upstream. > There's git/SVN integration, but I've not seen examples of how it can > be used by FreeBSD developers with SVN accounts; The Gitorious wiki page (http://wiki.freebsd.org/Gitorious) claims that git-svn can be successfully used with our SVN server with a command like: git svn commit-diff -m "git branch to svn" -rHEAD upstream/master work/ hwpmc_kcachegrind svn+ssh://svn.freebsd.org/base/user/fabient/svctest/ I have not tested this yet with path=/base/head, as it's release time and I suspect that people might get rather cranky if I mess things up too badly. I am definitely intending to test this approach once CURRENT is unfrozen, however, and document my experiences in the wiki. One of the downsides of using git-svn is that some things (e.g. "make sysent") expect the $FreeBSD$ in our header files to be expanded to something SVN-ey, but Git believes that it shouldn't munge source code: it's an immutable blob. So, when changing syscalls, one needs to check out syscalls.master using freebsd-subversion, copy it to the Git repo, run "make sysent" and then finally revert syscalls.master to what Git expects it to be (just "$FreeBSD$" at the top). There's a viable argument to be had here as to whether this is a Git problem or an assumption-that-the-script-makes problem, but it is a nit to be aware of. Jon -- Jonathan Anderson Research Student, Security Group Computer Laboratory University of Cambridge +44 (1223) 763747 jonathan.anderson@cl.cam.ac.uk From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 10:35:29 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1773C1065670; Fri, 26 Aug 2011 10:35:29 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9D3778FC1C; Fri, 26 Aug 2011 10:35:28 +0000 (UTC) Received: by gwb15 with SMTP id 15so3224452gwb.13 for ; Fri, 26 Aug 2011 03:35:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=9msJx1rizZG3fWhBWcySracoHs8psCFOxHXDxQtuTYU=; b=ctRt4M3bpNKe/sJG5VkHOiGpGiYiuxdgTkLJudClQjBNdpXE/rY3qelN/psxdT8E33 mpgNmEdNxxpUk3ePXnURjhIXiuSCUn4o9ng3VXTrrwRDtuUTXEdTrW0fzwIFH7O092Pr srAVC8WoZd+fLs6U0HcZkcpigj/K3MviV5XgU= MIME-Version: 1.0 Received: by 10.150.32.14 with SMTP id f14mr2071824ybf.201.1314354927502; Fri, 26 Aug 2011 03:35:27 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.145.21 with HTTP; Fri, 26 Aug 2011 03:35:27 -0700 (PDT) In-Reply-To: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Date: Fri, 26 Aug 2011 18:35:27 +0800 X-Google-Sender-Auth: diHR4NWUdOjoJgdJX0u0icPJQSE Message-ID: From: Adrian Chadd To: Jonathan Anderson Content-Type: text/plain; charset=ISO-8859-1 Cc: vadim_nuclight@mail.ru, mdf@freebsd.org, Robert Watson , freebsd-arch@freebsd.org Subject: Re: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 10:35:29 -0000 On 26 August 2011 18:04, Jonathan Anderson wrote: > The Gitorious wiki page (http://wiki.freebsd.org/Gitorious) claims > that git-svn can be successfully used with our SVN server with a > command like: [snip] Great, so which tree do I clone to be able to do this? :) Or is it expected that I'll simply run my own git tree? Do all of the git commit hashes get calculated in a predictable way when thirty different developers run a git to svn import? Ie, can people then push their git branches to some shared repository, so people can engage in git-like development mashing? adrian From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 11:23:58 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AC02106564A; Fri, 26 Aug 2011 11:23:58 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 6B1268FC12; Fri, 26 Aug 2011 11:23:58 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id D7C3346B06; Fri, 26 Aug 2011 07:23:57 -0400 (EDT) Date: Fri, 26 Aug 2011 12:23:57 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Adrian Chadd In-Reply-To: Message-ID: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: vadim_nuclight@mail.ru, mdf@freebsd.org, Jonathan Anderson , freebsd-arch@freebsd.org Subject: Re: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 11:23:58 -0000 On Fri, 26 Aug 2011, Adrian Chadd wrote: > On 26 August 2011 18:04, Jonathan Anderson > wrote: > >> The Gitorious wiki page (http://wiki.freebsd.org/Gitorious) claims that >> git-svn can be successfully used with our SVN server with a command like: > > [snip] > > Great, so which tree do I clone to be able to do this? :) Or is it expected > that I'll simply run my own git tree? Do all of the git commit hashes get > calculated in a predictable way when thirty different developers run a git > to svn import? Per my earlier comments, I think we've reached the juncture where a "blessed" svn2git export would be extremely helpful. > Ie, can people then push their git branches to some shared repository, so > people can engage in git-like development mashing? I suspect quite a bit will end up in github just due to its accessibility, but hosting our own is certainly also an option. In some sense, this strikes me as secondary to establishing some of the details about how to prepare patches, known nits, and having an authoritative origin. Robert From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 11:43:56 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73411106566B; Fri, 26 Aug 2011 11:43:56 +0000 (UTC) (envelope-from jra40@hermes.cam.ac.uk) Received: from ppsw-50.csi.cam.ac.uk (ppsw-50.csi.cam.ac.uk [131.111.8.150]) by mx1.freebsd.org (Postfix) with ESMTP id F32DB8FC08; Fri, 26 Aug 2011 11:43:55 +0000 (UTC) X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-qw0-f54.google.com ([209.85.216.54]:59593) by ppsw-50.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:465) with esmtpsa (PLAIN:jra40) (TLSv1:RC4-SHA:128) id 1QwupG-0004TF-sc (Exim 4.72) (return-path ); Fri, 26 Aug 2011 12:43:55 +0100 Received: by qwc9 with SMTP id 9so2495277qwc.13 for ; Fri, 26 Aug 2011 04:43:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.43.203 with SMTP id x11mr1331328qce.127.1314359033841; Fri, 26 Aug 2011 04:43:53 -0700 (PDT) Received: by 10.229.232.196 with HTTP; Fri, 26 Aug 2011 04:43:53 -0700 (PDT) In-Reply-To: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Date: Fri, 26 Aug 2011 12:43:53 +0100 Message-ID: From: Jonathan Anderson To: Robert Watson Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sender: Jonathan Anderson Cc: vadim_nuclight@mail.ru, mdf@freebsd.org, Adrian Chadd , freebsd-arch@freebsd.org Subject: Re: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 11:43:56 -0000 On 26 August 2011 12:23, Robert Watson wrote: > Per my earlier comments, I think we've reached the juncture where a > "blessed" svn2git export would be extremely helpful. > > [...] > > I suspect quite a bit will end up in github just due to its accessibility= , > but hosting our own is certainly also an option. =C2=A0In some sense, thi= s > strikes me as secondary to establishing some of the details about how to > prepare patches, known nits, and having an authoritative origin. Indeed, that's the beauty of Git, that the repositories will all play nicely together, no matter where they're hosted or what the "chain of custody" is, but they do need to come from the "right" source. I just hope that the consensus solidifies around the freebsd.git repo that your.org hosts, rather than the freebsd-head.git one that your.org, github and gitorious all have, because your.org's freebsd.git has *all* of the SVN branches, including releases, vendor branches, user projects, etc., whereas the others only have [variously] -CURRENT or a smattering of release branches. Jon --=20 Jonathan Anderson Research Student, Security Group Computer Laboratory University of Cambridge +44 (1223) 763747 jonathan.anderson@cl.cam.ac.uk From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 12:42:27 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71A11106564A; Fri, 26 Aug 2011 12:42:27 +0000 (UTC) (envelope-from jonathan.robert.anderson@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id CFC6D8FC0A; Fri, 26 Aug 2011 12:42:26 +0000 (UTC) Received: by qwc9 with SMTP id 9so2558615qwc.13 for ; Fri, 26 Aug 2011 05:42:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=5aFc33yBR4IdNcFWuG0Lh45gGwpsxZWzNpbRtIEcTzc=; b=S1VEKKJgBZWfJKR/wH4vjj+BbCSed/Oc4UZhl3HlGJB29TXlYc5RDzdGqb6mWeTA21 E5z5bJLRl9fhp87NP+JKX1mk5qOC/ZR9W8Hp3vhxFsQeXXtE6QwhSvHtFmrr9Ac4ujMU 0ZjgtBmpmYCrAI7O/HcUMZGCBSigzrl3x32m8= MIME-Version: 1.0 Received: by 10.229.64.146 with SMTP id e18mr1384395qci.165.1314360698315; Fri, 26 Aug 2011 05:11:38 -0700 (PDT) Sender: jonathan.robert.anderson@gmail.com Received: by 10.229.232.196 with HTTP; Fri, 26 Aug 2011 05:11:38 -0700 (PDT) In-Reply-To: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> Date: Fri, 26 Aug 2011 13:11:38 +0100 X-Google-Sender-Auth: 4vXe5IJPiK4Dp-2-E2RBIhAgjEs Message-ID: From: Jonathan Anderson To: Robert Watson Content-Type: text/plain; charset=UTF-8 Cc: vadim_nuclight@mail.ru, mdf@freebsd.org, Adrian Chadd , freebsd-arch@freebsd.org Subject: Re: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 12:42:27 -0000 [snip] By the way, I've put my thoughts about using Git for FreeBSD development up at http://wiki.freebsd.org/Git (which references the /GitConversion and /Gitorious pages). I'm very happy to be criticized re:poor Git usage, incomplete explanations and/or copious Wiki style horrors. I don't claim to be right. :) I do hope that this concrete brain dump will spark more discussion of the form "this specific thing that you do is silly, everybody ought to be doing X instead", leading to broad consensus around useful practices and other people distilling /Git to be a summary of "what you need to know if you want to start hacking FreeBSD using Git". Jon -- Jonathan Anderson jonathan@FreeBSD.org http://freebsd.org/~jonathan/ From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 18:28:47 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F061D106564A; Fri, 26 Aug 2011 18:28:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DA5BF8FC0C; Fri, 26 Aug 2011 18:28:45 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id VAA26090; Fri, 26 Aug 2011 21:28:43 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Qx190-000HZc-UW; Fri, 26 Aug 2011 21:28:42 +0300 Message-ID: <4E57E5D8.3010606@FreeBSD.org> Date: Fri, 26 Aug 2011 21:28:40 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0) Gecko/20110819 Thunderbird/6.0 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> In-Reply-To: <201108250945.24606.jhb@freebsd.org> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Hans Petter Selasky , John Baldwin Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 18:28:47 -0000 I looked a little bit through the code trying to get some facts about the locking in tty/syscons/kbd area (with respect to input, obviously). I would like to ask you to please double-check my observations and also to answer some questions that follow from the observations, if possible. So, first of all, I see almost no explicit Giant locking in the code related to tty/syscons/kbd subsystems/drivers/layers. By kbd I mean kbd, kbdmux, vkbd and atkbdc, but not ukbd, which I consider as a special case. The code seems to get the locking via the following means: - kbd/syscons cdevs are marked as needing the Giant - Giant-protected variant of callouts is used - wherever taskqueues are used they are used with the Giant - console vtys are allocated using tty_alloc_mutex(Giant) The only case where the Giant is acquired explicitly is sckbdevent() in the syscons driver. Also, and maybe this is beyond the topic at hand, I do not see any other locking that would protect internal states of syscons and atkbd* drivers. Of course, they are littered with spl* calls, but those are NOPs now. Another observation is that when the kernel code calls into syscons code (kern_cons.c) it doesn't acquire the Giant. It seems that the kernel calls syscons input routines only in some very special situations like very early boot (pause after each line option), late shutdown stage ("press any key to...") and ddb command prompt. Based on the above I make some further conclusions. The Giant is held when upper levels call into syscons and kbd drivers and the calls are originating from userland. The Giant is held when the kbd drivers call into syscons via sckbdevent (as kb_callback). Thus, I think that the internal state of syscons is protected by the Giant alone. The only case where it is not protected and where the protection would probably be unneeded or even harmful is when the kernel calls into the syscons in the special situations. I do not see how internal state of kbd drivers is protected from races between interrupts and calls from upper layers. The only possibility that I see is that kbd interrupt handlers do not directly change the driver state but rather "call back" into the driver via sckbdevent(). In this case the state is always protected by the Giant in normal situations. And it should not need to be protected in the special situations, because in those there should be neither concurrent access nor interrupts. Again, I think that when the kernel calls the syscons inputs routines, sc_cngetc actually, then the syscons puts a kbd driver into polling mode. So further calls should be safe from races with a kbd interrupt handler. Hmm, but it's actually the kbd poll(TRUE) call itself that can be racy-ish... As a sidenote, I think that ukbd is not used during early boot, because we do not have any special code for early USB keyboard discovery and typically ukbd attaches quite late during boot (sometimes even too late, at least in the past). And now, provided that no other kbd driver explicitly obtains the Giant and thus depends on upper levels doing the right thing - either locking the Giant or knowing that it needs not to be locked, I have to ask why ukbd needs to explicitly take the Giant. Does ukbd interact with other layers/subsystems substantially differently from other kbd drivers? Does ukbd needs the Giant because of the rest of the USB subsystem? Does it need the Giant inherently because of how it accesses its internal state? Thank you very much in advance! P.S. I will greatly appreciate if each of my statements about the FreeBSD code could be annotated with the correct/incorrect comment. I do realize how much effort that is. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 18:56:38 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 37AB0106564A; Fri, 26 Aug 2011 18:56:38 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 896388FC12; Fri, 26 Aug 2011 18:56:37 +0000 (UTC) Received: by fxe4 with SMTP id 4so3614938fxe.13 for ; Fri, 26 Aug 2011 11:56:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=wgnGXAKNAXIXxk2hU3C9AR0JA4X55GC7xQgW5ldJJnA=; b=Z0aNBsWa7YD7Pwu7+tMsWrUwug7Yu1udy5A/LMLBnBu2K/EGOGWoAO71buOvPHOeqC rLz40V6chtFWuspTpz85dCRMYnH5fzA6MnDX1jUBiAKhy3VjIU5nIcLWZOcSeARiEz6v rWTPi2qQc5K5Ez1DB0duOUAg7oKlpJXpSjnyA= Received: by 10.223.102.10 with SMTP id e10mr2063347fao.55.1314383599038; Fri, 26 Aug 2011 11:33:19 -0700 (PDT) Received: from localhost (lan-78-157-92-5.vln.skynet.lt [78.157.92.5]) by mx.google.com with ESMTPS id l22sm1536771fam.13.2011.08.26.11.33.16 (version=SSLv3 cipher=OTHER); Fri, 26 Aug 2011 11:33:17 -0700 (PDT) Date: Fri, 26 Aug 2011 21:31:31 +0300 From: Gleb Kurtsou To: Jonathan Anderson Message-ID: <20110826183130.GA40586@tops> References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: vadim_nuclight@mail.ru, mdf@freebsd.org, Adrian Chadd , Robert Watson , freebsd-arch@freebsd.org Subject: Re: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 18:56:38 -0000 On (26/08/2011 12:43), Jonathan Anderson wrote: > On 26 August 2011 12:23, Robert Watson wrote: > > Per my earlier comments, I think we've reached the juncture where a > > "blessed" svn2git export would be extremely helpful. > > > > [...] > > > > I suspect quite a bit will end up in github just due to its accessibility, > > but hosting our own is certainly also an option.  In some sense, this > > strikes me as secondary to establishing some of the details about how to > > prepare patches, known nits, and having an authoritative origin. > > Indeed, that's the beauty of Git, that the repositories will all play > nicely together, no matter where they're hosted or what the "chain of > custody" is, but they do need to come from the "right" source. > > I just hope that the consensus solidifies around the freebsd.git repo > that your.org hosts, rather than the freebsd-head.git one that > your.org, github and gitorious all have, because your.org's > freebsd.git has *all* of the SVN branches, including releases, vendor > branches, user projects, etc., whereas the others only have > [variously] -CURRENT or a smattering of release branches. I'd also prefer your.org style repository with all branches. your.org branch layout is much better than http://gitorious.org/freebsd/freebsd The only thing that bothers me is that no repository includes full user names, hopefully it would be fixed in official git repo. Jonathan, could you update /Git wiki page with an example of git clone specifying only head branch and how to add another branch later. That would make local repository copy smaller, pull faster and 'git branch -r' output much shorter. Thanks, Gleb. > > > Jon > -- > Jonathan Anderson > > Research Student, Security Group > Computer Laboratory > University of Cambridge > > +44 (1223) 763747 > jonathan.anderson@cl.cam.ac.uk From owner-freebsd-arch@FreeBSD.ORG Fri Aug 26 19:25:04 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8C73106566C; Fri, 26 Aug 2011 19:25:04 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D3B678FC08; Fri, 26 Aug 2011 19:25:03 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id WAA26463; Fri, 26 Aug 2011 22:25:02 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Qx21V-000HbO-Pp; Fri, 26 Aug 2011 22:25:01 +0300 Message-ID: <4E57F30C.4000400@FreeBSD.org> Date: Fri, 26 Aug 2011 22:25:00 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0) Gecko/20110819 Thunderbird/6.0 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> <4E57E5D8.3010606@FreeBSD.org> In-Reply-To: <4E57E5D8.3010606@FreeBSD.org> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Hans Petter Selasky , John Baldwin Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2011 19:25:05 -0000 on 26/08/2011 21:28 Andriy Gapon said the following: [snip] > Another observation is that when the kernel code calls into syscons code > (kern_cons.c) it doesn't acquire the Giant. It seems that the kernel calls > syscons input routines only in some very special situations like very early boot > (pause after each line option), late shutdown stage ("press any key to...") and > ddb command prompt. [snip] > Hmm, but it's actually the kbd poll(TRUE) call itself that can be racy-ish... As I was going to argue with myself that there actually should not be any race here, because interrupts should already be disabled in the special cases, I suddenly realized that I forgot about yet another special case, which is actually not special enough - the mountroot prompt. And I think that this is the case which is the most troublesome comparing to all other cases. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 00:44:10 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 8EFDF1065675; Sat, 27 Aug 2011 00:44:10 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-198-245.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 4698414E5B0; Sat, 27 Aug 2011 00:43:34 +0000 (UTC) Message-ID: <4E583DB5.1090601@FreeBSD.org> Date: Fri, 26 Aug 2011 17:43:33 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0) Gecko/20110824 Thunderbird/6.0 MIME-Version: 1.0 To: Gleb Kurtsou References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> <20110826183130.GA40586@tops> In-Reply-To: <20110826183130.GA40586@tops> X-Enigmail-Version: undefined OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: mdf@freebsd.org, Adrian Chadd , Jonathan Anderson , vadim_nuclight@mail.ru, Robert Watson , freebsd-arch@freebsd.org Subject: Re: Official git export X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 00:44:10 -0000 On 08/26/2011 11:31, Gleb Kurtsou wrote: > Jonathan, could you update /Git wiki page with an example of git clone > specifying only head branch and how to add another branch later. That > would make local repository copy smaller, pull faster and > 'git branch -r' output much shorter. I would be more inclined to look at git if there were easily available instructions of this nature. FWIW, Doug (thinking I'm representative of enough similarly-minded people to make saying it out loud worthwhile) -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 01:55:49 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CFDA106566B; Sat, 27 Aug 2011 01:55:49 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0A00E8FC13; Sat, 27 Aug 2011 01:55:48 +0000 (UTC) Received: by gyd10 with SMTP id 10so4033049gyd.13 for ; Fri, 26 Aug 2011 18:55:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=MLiDUt6yYzITlHIE75aa5de8mOmB3wF3PW1X1YDxyu4=; b=BiVUqOnJfTHSGTcM9AS8YgIOmSdDIZ6SBE0Vp4F5ofkU0t3KF6KyNdsUaR9e1Gs1Dt Hk+Mj9Dvu6TXrWLYdkj1oYninf0hLBxI2c10yiN8mkuzIwK4hPe7FWffvBNnjsvAa3nN M5vKbbWHQWyV2Y+pcG1TcGwg5IHFcGGZHlDCM= MIME-Version: 1.0 Received: by 10.236.181.131 with SMTP id l3mr10918865yhm.44.1314408378549; Fri, 26 Aug 2011 18:26:18 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.236.102.147 with HTTP; Fri, 26 Aug 2011 18:26:18 -0700 (PDT) In-Reply-To: <20110826183130.GA40586@tops> References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> <20110826183130.GA40586@tops> Date: Fri, 26 Aug 2011 18:26:18 -0700 X-Google-Sender-Auth: gKDmTNwRIgAMxR2oZDyq0cyUiio Message-ID: From: Artem Belevich To: Gleb Kurtsou Content-Type: text/plain; charset=ISO-8859-1 Cc: mdf@freebsd.org, Adrian Chadd , Jonathan Anderson , vadim_nuclight@mail.ru, Robert Watson , freebsd-arch@freebsd.org Subject: Re: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 01:55:49 -0000 Hi, On Fri, Aug 26, 2011 at 11:31 AM, Gleb Kurtsou wrote: > Jonathan, could you update /Git wiki page with an example of git clone > specifying only head branch and how to add another branch later. That > would make local repository copy smaller, pull faster and > 'git branch -r' output much shorter. Here you go: $ mkdir small-clone $ cd small-clone $ git init # Tell git to fetch only master (AKA head) and stable/8 branches $ git remote add -t master -t stable/8 origin git://git.freebsd.your.org/freebsd.git $ git fetch origin # fetches about 1.6M objects remote: Counting objects: 1600952, done. remote: Compressing objects: 100% (418377/418377), done. remote: Total 1600952 (delta 1227540), reused 1524707 (delta 1159458) Receiving objects: 100% (1600952/1600952), 632.85 MiB | 3.12 MiB/s, done. Resolving deltas: 100% (1227540/1227540), done. From git://git.freebsd.your.org/freebsd * [new branch] master -> origin/master * [new branch] stable/8 -> origin/stable/8 # Add stable/7 branch and fetch it $ git remote set-branches origin --add stable/7 $ git fetch origin remote: Counting objects: 35713, done. remote: Compressing objects: 100% (11359/11359), done. remote: Total 27595 (delta 20679), reused 22440 (delta 16043) Receiving objects: 100% (27595/27595), 14.40 MiB | 1.93 MiB/s, done. Resolving deltas: 100% (20679/20679), completed with 2329 local objects. From git://git.freebsd.your.org/freebsd * [new branch] stable/7 -> origin/stable/7 $ git branch -r origin/HEAD -> origin/head origin/master origin/stable/7 origin/stable/8 Practically it does not save you all that much on repo size or the time to fetch stuff. Complete clone is ~700M/1.9M objects and head+stable/8 clone is only about 10% smaller. If you really want to have smaller repo you will need to trip repo history. I'm not sure yet how to do that. --Artem From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 01:58:40 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D092106566B; Sat, 27 Aug 2011 01:58:40 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU [18.9.25.15]) by mx1.freebsd.org (Postfix) with ESMTP id BEC378FC14; Sat, 27 Aug 2011 01:58:39 +0000 (UTC) X-AuditID: 1209190f-b7b44ae000000a24-7b-4e584b2e2fa5 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP id 33.A1.02596.E2B485E4; Fri, 26 Aug 2011 21:41:02 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id p7R1hbOQ009461; Fri, 26 Aug 2011 21:43:38 -0400 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id p7R1hYQl026865 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 26 Aug 2011 21:43:37 -0400 (EDT) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id p7R1hYCf003223; Fri, 26 Aug 2011 21:43:34 -0400 (EDT) Date: Fri, 26 Aug 2011 21:43:34 -0400 (EDT) From: Benjamin Kaduk To: Robert Watson In-Reply-To: Message-ID: References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBIsWRmVeSWpSXmKPExsUixCmqravnHeFnsOq9pcXs6dOYLPoOzWZ0 YPKY8Wk+SwBjFJdNSmpOZllqkb5dAldG+6KDTAWLmCuuvprI0sB4mqmLkZNDQsBE4vnBmewQ tpjEhXvr2boYuTiEBPYxSrSc/s0M4WxglGhYvp0RwjnAJNG/bxIrSIuQQAOjxMOTFiA2i4C2 xNEzL8BGsQmoSMx8s5ENxBYRUJe4ch5kLAcHs4CMxJzX3iBhYYEoia+zz7KA2JwCLhK7u76C 2bwCDhJdn4+wQOz6yyjx8dxnsISogI7E6v1ToIoEJU7OfAJmMwtYSpz7c51tAqPgLCSpWUhS CxiZVjHKpuRW6eYmZuYUpybrFicn5uWlFuma6OVmluilppRuYgQHqCT/DsZvB5UOMQpwMCrx 8M7wiPATYk0sK67MPcQoycGkJMob4wUU4kvKT6nMSCzOiC8qzUktPsQowcGsJMJ7fXO4nxBv SmJlVWpRPkxKmoNFSZy3cYeDn5BAemJJanZqakFqEUxWhoNDSYL3uhPQUMGi1PTUirTMnBKE NBMHJ8hwHqDhisCIFuItLkjMLc5Mh8ifYlSUEuf1AEkIgCQySvPgemEJ5BWjONArwrzaIFU8 wOQD1/0KaDAT0GAVR5Cri0sSEVJSDYz8Z3/+NZc+vmc16/E/VyrOptYxXZrA4bbjV5nVkYOp Hy7kBm4XMGBWc715651bVmdrvj7/hhUbmrdu+l1140znq+fHC88qlp7ki9iR+E7BzPdea8Xn a9wZyidFQ3mSTaxfZYgmPl/4lKmvfPdHbfEvS4Ru1rMmeh84dnzN1vADLw+t0LoQ2LpWiaU4 I9FQi7moOBEAU8RTePsCAAA= Cc: freebsd-arch@freebsd.org Subject: Re: Removing Giant from VFS in 10.0 (was: Re: skipping locks, mutex_owned, usb) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 01:58:40 -0000 On Fri, 26 Aug 2011, Robert Watson wrote: > We also need to start announcing this early in the 10.0 cycle so that > third-party file system developers for FreeBSD -- especially anyone > interested in things like OpenAFS and Fuse, can do appropriate updates there > as well. For what it's worth, OpenAFS already passes MNTK_MPSAFE /* solid steel */, with what I expect is nearly-correct locking. -Ben From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 01:59:30 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 74EBF106564A; Sat, 27 Aug 2011 01:59:30 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-yi0-f54.google.com (mail-yi0-f54.google.com [209.85.218.54]) by mx1.freebsd.org (Postfix) with ESMTP id E43068FC0A; Sat, 27 Aug 2011 01:59:29 +0000 (UTC) Received: by yib19 with SMTP id 19so2475495yib.13 for ; Fri, 26 Aug 2011 18:59:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=9yxNMODgcd+bEPXdlrw5udqGT1Z6lWNhbuwWACXbOWc=; b=rbrfrEfg1VrVYuiSXmBOJsNyVMryAazZLbkq3Kg3iKA6JDLQWTSv6K9DaN9QqhZpiZ U/WKYxa4ggaO4EuAt0wmop0DqAFHBB9L0OS9F8FIB2SArnIeuFvXOPyp55h+XjNaXY0a TNmRb0HW7JLdgJHd1WV6uNF5Oi8VRvfdlOrjc= MIME-Version: 1.0 Received: by 10.150.229.12 with SMTP id b12mr2788185ybh.30.1314410369238; Fri, 26 Aug 2011 18:59:29 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.145.21 with HTTP; Fri, 26 Aug 2011 18:59:29 -0700 (PDT) In-Reply-To: <4E583DB5.1090601@FreeBSD.org> References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> <20110826183130.GA40586@tops> <4E583DB5.1090601@FreeBSD.org> Date: Sat, 27 Aug 2011 09:59:29 +0800 X-Google-Sender-Auth: NT47y75cgfv60tVMKnwcpQ-sHAQ Message-ID: From: Adrian Chadd To: Doug Barton Content-Type: text/plain; charset=ISO-8859-1 Cc: mdf@freebsd.org, Gleb Kurtsou , Jonathan Anderson , vadim_nuclight@mail.ru, Robert Watson , freebsd-arch@freebsd.org Subject: Re: Official git export X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 01:59:30 -0000 On 27 August 2011 08:43, Doug Barton wrote: > On 08/26/2011 11:31, Gleb Kurtsou wrote: >> Jonathan, could you update /Git wiki page with an example of git clone >> specifying only head branch and how to add another branch later. That >> would make local repository copy smaller, pull faster and >> 'git branch -r' output much shorter. > > I would be more inclined to look at git if there were easily available > instructions of this nature. I don't think git "does" this, does it? Adrian From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 02:00:51 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CFEDE1065673; Sat, 27 Aug 2011 02:00:51 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 48FD88FC0A; Sat, 27 Aug 2011 02:00:51 +0000 (UTC) Received: by ywo32 with SMTP id 32so4033490ywo.13 for ; Fri, 26 Aug 2011 19:00:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=uebIYdMZW5/fw7CaXi2uVOf+YOEWPhw4O+GGqG1LYy8=; b=CMJSTvXL8v9vvHG1Z6F6tnGhicfJmeNhcFqQFli4yW8aSROO1R47uL+4HvNd6zlnk8 EI74TNA1EM4sPk1kXxu6X8nnbvxuyBbgf9TQspy/ds0+JHzc0iV0q6K0UPJi6L3PGmf3 c62jzw+MoQVjbXcbgU3fOkbryMxPBrODJMjq8= MIME-Version: 1.0 Received: by 10.150.238.17 with SMTP id l17mr2724594ybh.144.1314410450739; Fri, 26 Aug 2011 19:00:50 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.145.21 with HTTP; Fri, 26 Aug 2011 19:00:50 -0700 (PDT) In-Reply-To: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> <20110826183130.GA40586@tops> <4E583DB5.1090601@FreeBSD.org> Date: Sat, 27 Aug 2011 10:00:50 +0800 X-Google-Sender-Auth: Hk2pwoPbYyUjjd4cW9p1YM2Hj6g Message-ID: From: Adrian Chadd To: Doug Barton Content-Type: text/plain; charset=ISO-8859-1 Cc: mdf@freebsd.org, Gleb Kurtsou , Jonathan Anderson , vadim_nuclight@mail.ru, Robert Watson , freebsd-arch@freebsd.org Subject: Re: Official git export X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 02:00:51 -0000 .. nope, it does. I just saw the other reply. Cool! Adrian From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 08:44:34 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B860106566C for ; Sat, 27 Aug 2011 08:44:34 +0000 (UTC) (envelope-from hselasky@freebsd.org) Received: from swip.net (mailfe03.c2i.net [212.247.154.66]) by mx1.freebsd.org (Postfix) with ESMTP id 0A9C58FC16 for ; Sat, 27 Aug 2011 08:44:33 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=yVKV3zusvCapyMfYJBNW2j35FMEuTKq6vh/tt/1L5+g= c=1 sm=1 a=SvYTsOw2Z4kA:10 a=ni9ZY7ZFbhgA:10 a=dBRESv0yCI8A:10 a=8nJEP1OIZ-IA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=I-8bbMwgEVfJMHRZjpEA:9 a=wPNLvfGTeEIA:10 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe03.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 3831585; Sat, 27 Aug 2011 10:34:29 +0200 Received-SPF: softfail receiver=mailfe03.swip.net; client-ip=188.126.198.129; envelope-from=hselasky@freebsd.org From: Hans Petter Selasky To: freebsd-arch@freebsd.org Date: Sat, 27 Aug 2011 10:31:56 +0200 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4E53986B.5000804@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> <4E57E5D8.3010606@FreeBSD.org> In-Reply-To: <4E57E5D8.3010606@FreeBSD.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108271031.56642.hselasky@freebsd.org> Cc: Andriy Gapon Subject: Re: skipping locks, mutex_owned, usb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 08:44:34 -0000 On Friday 26 August 2011 20:28:40 Andriy Gapon wrote: > Does ukbd needs the Giant because of the rest of the USB subsystem? No. It can use any other mutex or its own mutex. --HPS From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 09:57:03 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA9521065674; Sat, 27 Aug 2011 09:57:03 +0000 (UTC) (envelope-from rwatson@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id AA95D8FC16; Sat, 27 Aug 2011 09:57:03 +0000 (UTC) Received: from [192.168.2.112] (host81-151-180-177.range81-151.btcentralplus.com [81.151.180.177]) by cyrus.watson.org (Postfix) with ESMTPSA id E531746B49; Sat, 27 Aug 2011 05:57:01 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <4E583DB5.1090601@FreeBSD.org> Date: Sat, 27 Aug 2011 10:56:59 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <35765857-1314243257-cardhu_decombobulator_blackberry.rim.net-329610575-@b2.c15.bise7.blackberry> <20110826183130.GA40586@tops> <4E583DB5.1090601@FreeBSD.org> To: Doug Barton X-Mailer: Apple Mail (2.1084) Cc: mdf@freebsd.org, Gleb Kurtsou , Jonathan Anderson , Adrian Chadd , vadim_nuclight@mail.ru, freebsd-arch@freebsd.org Subject: Re: Official git export X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 09:57:03 -0000 On 27 Aug 2011, at 01:43, Doug Barton wrote: > On 08/26/2011 11:31, Gleb Kurtsou wrote: >> Jonathan, could you update /Git wiki page with an example of git = clone >> specifying only head branch and how to add another branch later. That >> would make local repository copy smaller, pull faster and >> 'git branch -r' output much shorter. >=20 > I would be more inclined to look at git if there were easily available > instructions of this nature. > ... > Doug (thinking I'm representative of enough similarly-minded people to > make saying it out loud worthwhile) As with all revision control systems, there are lots of ways to use git. = I think there's huge value in documenting an authoritative "recommended" = way to use git with FreeBSD. Other very similar projects to ours do = exactly this -- for example, LLVM: http://llvm.org/docs/GettingStarted.html#git_mirror And Chromium: http://code.google.com/p/chromium/wiki/UsingGit Note that Chromium even wraps it up in a script to ensure the details = are done right, and has notes on issues with getting Subversion = auto-props right, etc. The above both look a fair amount like Jon's draft notes on our wiki, in = fact :-). Robert= From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 09:59:05 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD4881065672 for ; Sat, 27 Aug 2011 09:59:05 +0000 (UTC) (envelope-from rwatson@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 858C98FC0C for ; Sat, 27 Aug 2011 09:59:05 +0000 (UTC) Received: from [192.168.2.112] (host81-151-180-177.range81-151.btcentralplus.com [81.151.180.177]) by cyrus.watson.org (Postfix) with ESMTPSA id 84A0E46B06; Sat, 27 Aug 2011 05:59:04 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: Date: Sat, 27 Aug 2011 10:59:03 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> To: Benjamin Kaduk X-Mailer: Apple Mail (2.1084) Cc: freebsd-arch@freebsd.org Subject: Re: Removing Giant from VFS in 10.0 (was: Re: skipping locks, mutex_owned, usb) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 09:59:05 -0000 On 27 Aug 2011, at 02:43, Benjamin Kaduk wrote: > On Fri, 26 Aug 2011, Robert Watson wrote: >=20 >> We also need to start announcing this early in the 10.0 cycle so that = third-party file system developers for FreeBSD -- especially anyone = interested in things like OpenAFS and Fuse, can do appropriate updates = there as well. >=20 > For what it's worth, OpenAFS already passes MNTK_MPSAFE /* solid steel = */, with what I expect is nearly-correct locking. Excellent! (Although I guess OpenAFS internally has the moral equivalent of a Giant = lock that protects its own structures, but that's an entirely = independent problem that the OpenAFS community is already interested = in?) Robert= From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 12:24:13 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A6A81065675 for ; Sat, 27 Aug 2011 12:24:13 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 33CAF8FC1C for ; Sat, 27 Aug 2011 12:24:12 +0000 (UTC) Received: by wwi36 with SMTP id 36so4343828wwi.31 for ; Sat, 27 Aug 2011 05:24:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=RDMNnosI9KniBdB7WF2tbR4EhuP2E1SB7goIvMgHsRw=; b=PNLzFfRMAjWLKNzPDwkyP3ROoXp26csdPhZfcnefsmUYUfNMmCTBN5sblVFwcKVnf2 mLCwVx+G4VNdj/PA5qEXQtnocmGD5PrY+XZ+qsIxbqD9mPWEtzEL8NldKXCaCtpv/VUR 8AU5dbg1gTv08H0efkzW/MekPq6B23YdoedzM= MIME-Version: 1.0 Received: by 10.227.72.200 with SMTP id n8mr1083196wbj.19.1314446450094; Sat, 27 Aug 2011 05:00:50 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.227.206.139 with HTTP; Sat, 27 Aug 2011 05:00:50 -0700 (PDT) Date: Sat, 27 Aug 2011 14:00:50 +0200 X-Google-Sender-Auth: vbAIcujJKXBpup-9FAoZdXFwgU4 Message-ID: From: Attilio Rao To: freebsd-arch@freebsd.org, freebsd-current@freebsd.org, FreeBSD FS Content-Type: text/plain; charset=UTF-8 Cc: Robert Watson , Konstantin Belousov Subject: Removal of Giant from the VFS layer for 10.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 12:24:13 -0000 [ Sorry for cross-posting, but I included -arch@ for technical discussion, -current@ for reaching the wider audience and -fs@ for the relevance of the matter.] During the last years a lot of effort by several developers happened in order to reduce Giant influence over the entire kernel. The VFS layer didn't make an exception, as many several tasks have been completed along the years, including fine-grained locking for vnodes lifecycle, fine-grained locking of the VFS structure (mount), fine-grained locking of specific filesystems (UFS, NFS, etc.) and several locking improvements to surrounding subsystem (buffer cache, filedesc objects, VM layer, etc.). While FreeBSD did pretty well so far, a major push is still needed in order to completely remove Giant from our VFS and buffer cache subsystems. At the present time, the biggest problem is that there are still filesystems which are not properly fine-grained locked, relying on Giant for assuring atomicity. It is time to make an decision for them, in order to aim for a Giant-less VFS in our next release. With the aid of kib and rwatson I made a roughly outlined plan about what is left to do in order to have all the filesystems locked (or eventually dropped) before 10.0) and is summarized here: http://wiki.freebsd.org/NONMPSAFE_DEORBIT_VFS As you can note from the page, the plan is thought to be 18 months long, including time for developers to convert all our filesystems and let thirdy-party producers do the same with their proprietary filesystems. Also the introduction (and later on removal) of VFS_GIANT_COMPATIBILITY is thought to stress-out the presence of not-yet MPSAFE filesystems used by consumers and force a proactive action. As you can note from the page, the list of filesystems to be converted is small and well contained, but there are some edge cases that really concerns me, like ntfs and smbfs. I took over leadership of ntfs, but if someone is willing to override myself, please just drop an e-mail and I'll happilly hand over someone else. About smbfs, I really think this is really the key filesystem we should fix in the list and it is time for someone to step up and do the job (including also locking and reworking netsmb). I knew there was a Google SoC going on this topic, but didn't have further updates to the matter in weeks. Ideally, after all the filesystems are locked, what should happen is to remove all Giant reference from the VFS, as kib's patch present in the wiki page. If some filesystem is still left for the 1st Semptember of next year, it is going to be disconnected from the tree along with Giant axing. As the locking of filesystems progresses, we can create subsections for each filesystems including technical notes on the matter. So fare there is none because the effort is still not started. The page is also thought to contain technical notes on how to operate the locking of filesystems, in more general way. I added the msdosfs example as a reference but other cases may have different problems. However, as the state of all the filesystems listed in the black page is a bit unknown, I'd suggest you to first make it work stable and just in the end work on locking. Also, please remind that locking doesn't need to be perfect at the first time, it is enough to bring the filesystem out of the Giant influence intially. Of course, for key filesystems (smbfs in primis) I'd expect to have full fine-grained locking support at some point. During the 18 months timeframe I'll send some reminder and status updates to these lists (monthly or bi-monthly). If there is anything else you want to discuss about this plan, don't hesitate to contact me. There is one last thing I want to stress out: this type of activities rely a lot on the audience to step up and make the job. Please don't expect someone else to fix the filesystem for you, but be proactive as much as you can, offering quality time for development, testing and reviews. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 15:07:19 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2DDD5106564A; Sat, 27 Aug 2011 15:07:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id BDA0A8FC0C; Sat, 27 Aug 2011 15:07:18 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p7RF4UMu001658 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 27 Aug 2011 18:04:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p7RF4Utv034565; Sat, 27 Aug 2011 18:04:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p7RF4Uqr034564; Sat, 27 Aug 2011 18:04:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 27 Aug 2011 18:04:30 +0300 From: Kostik Belousov To: Attilio Rao Message-ID: <20110827150430.GI17489@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LzERIFExplvR0PTW" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD FS , freebsd-current@freebsd.org, Robert Watson , freebsd-arch@freebsd.org Subject: Re: Removal of Giant from the VFS layer for 10.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 15:07:19 -0000 --LzERIFExplvR0PTW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Aug 27, 2011 at 02:00:50PM +0200, Attilio Rao wrote: > [ Sorry for cross-posting, but I included -arch@ for technical > discussion, -current@ for reaching the wider audience and -fs@ for the > relevance of the matter.] >=20 > During the last years a lot of effort by several developers happened > in order to reduce Giant influence over the entire kernel. > The VFS layer didn't make an exception, as many several tasks have > been completed along the years, including fine-grained locking for > vnodes lifecycle, fine-grained locking of the VFS structure (mount), > fine-grained locking of specific filesystems (UFS, NFS, etc.) and > several locking improvements to surrounding subsystem (buffer cache, > filedesc objects, VM layer, etc.). >=20 > While FreeBSD did pretty well so far, a major push is still needed in > order to completely remove Giant from our VFS and buffer cache > subsystems. > At the present time, the biggest problem is that there are still > filesystems which are not properly fine-grained locked, relying on > Giant for assuring atomicity. It is time to make an decision for them, > in order to aim for a Giant-less VFS in our next release. The scope of the project should be made slightly more concrete. If you do not use a non-mpsafe fs, then VFS does not acquire Giant. This is true at least for stable/8 and HEAD kernels, might be also true for stable/7, but I do not remember for sure. The aim of the project is to remove compatibility shims that conditionally acquire Giant on the as-needed basis to allow non-mpsafe filesystems to operate still under the usual locking regime. In other words, the project will not make anything much faster or scalable, but to remove some quite large amount of the crafty code from our VFS, which is, unfortunately, not known for the very clean interfaces. --LzERIFExplvR0PTW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk5ZB34ACgkQC3+MBN1Mb4hZtACfQ0FFi0h+ySq6/yqLdaa8TKb1 l7MAnju58Ptqb8WXmYsHvziA3XwRusP/ =yjMg -----END PGP SIGNATURE----- --LzERIFExplvR0PTW-- From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 15:59:41 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8C2C106564A for ; Sat, 27 Aug 2011 15:59:41 +0000 (UTC) (envelope-from ulrich@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 00D818FC12 for ; Sat, 27 Aug 2011 15:59:40 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id p7RFxYqO074601 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 27 Aug 2011 17:59:34 +0200 (CEST) (envelope-from ulrich@spoerlein.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1314460775; bh=F8dcqrn2NEMnxJP+S06hm0LXXOhL7KbyDafZBO99Y6Q=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Transfer-Encoding:In-Reply-To; b=kVorHtHNMUATadyUvw+6uf+RLCXQd2z2Qtzf26ddYvhXGLDdUYK4UoWHZ9uF+ZNPI W7Jpa3HK/8irvoC3+wos5qGfnYFKEunD7fnSqkipumv09++kse4NXQVk0ajgT9kHD6 oGN7760yMiR1m5XxFYjEwV4LYiNDJmjgNTFocxkA= Date: Sat, 27 Aug 2011 17:59:34 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Gleb Kurtsou Message-ID: <20110827155934.GQ18091@acme.spoerlein.net> Mail-Followup-To: Gleb Kurtsou , Jonathan Anderson , vadim_nuclight@mail.ru, mdf@freebsd.org, Adrian Chadd , Robert Watson , freebsd-arch@freebsd.org References: <20110826183130.GA40586@tops> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20110826183130.GA40586@tops> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: mdf@freebsd.org, Adrian Chadd , Jonathan Anderson , vadim_nuclight@mail.ru, Robert Watson , freebsd-arch@freebsd.org Subject: Re: Official git export (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 15:59:41 -0000 On Fri, 2011-08-26 at 21:31:31 +0300, Gleb Kurtsou wrote: > On (26/08/2011 12:43), Jonathan Anderson wrote: > > On 26 August 2011 12:23, Robert Watson wrote: > > > Per my earlier comments, I think we've reached the juncture where a > > > "blessed" svn2git export would be extremely helpful. > > > > > > [...] > > > > > > I suspect quite a bit will end up in github just due to its accessibility, > > > but hosting our own is certainly also an option.  In some sense, this > > > strikes me as secondary to establishing some of the details about how to > > > prepare patches, known nits, and having an authoritative origin. > > > > Indeed, that's the beauty of Git, that the repositories will all play > > nicely together, no matter where they're hosted or what the "chain of > > custody" is, but they do need to come from the "right" source. > > > > I just hope that the consensus solidifies around the freebsd.git repo > > that your.org hosts, rather than the freebsd-head.git one that > > your.org, github and gitorious all have, because your.org's > > freebsd.git has *all* of the SVN branches, including releases, vendor > > branches, user projects, etc., whereas the others only have > > [variously] -CURRENT or a smattering of release branches. The key part here is svn integration. git-svn has this, but it does not cope with the huge history we have in the tree. Furthermore git-svn can/should be run be the individual committers and focusing on the main branch just makes the most sense. Especially, considering that you shouldn't use git-svn to merge changes from head to stable/* (blame the fucked up SVN merge mechanism). Thus, it's best to not even give committers the stable branches in their git trees, lest they mess up the svn repo. And it's really easy to push hundreds of commits from git to svn with one command. And we all make mistakes ... > I'd also prefer your.org style repository with all branches. your.org > branch layout is much better than http://gitorious.org/freebsd/freebsd Please use http://gitorious.org/freebsd/freebsd-head if you want a committer to actually take your patches and submit them into svn (using git-svn). > The only thing that bothers me is that no repository includes full user > names, hopefully it would be fixed in official git repo. No. I really would like to see the full names too, and actually put a lot of work into getting the correct names into the git repos in the first place. Alas it is too fragile and will break too often as the login -> full name mapping is not fixed (yay! we still get new people!) As part of using git-svn to push stuff upstream, it will first download and convert the svn head, rebase your patches and then send them as incremental commits. The "download and convert" part would require that each and every committer has the same, up-to-date, list of loginnames to email-addresses. This will break and the benefit is far too little anyway. Should a full conversion be done at some point (ha! last I heard our re@ folks are still using CVS ...), this can be reconsidered and fixed up easily. > Jonathan, could you update /Git wiki page with an example of git clone > specifying only head branch and how to add another branch later. That > would make local repository copy smaller, pull faster and > 'git branch -r' output much shorter. I'm fully to blame for the lack of documentation on a working git workflow. I shall review what's in the Wiki and come up with an overview of what' what. Cheers, Uli From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 18:08:33 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3B74106566C for ; Sat, 27 Aug 2011 18:08:33 +0000 (UTC) (envelope-from giffunip@tutopia.com) Received: from nm16-vm0.bullet.mail.sp2.yahoo.com (nm16-vm0.bullet.mail.sp2.yahoo.com [98.139.91.210]) by mx1.freebsd.org (Postfix) with SMTP id ADFCA8FC0C for ; Sat, 27 Aug 2011 18:08:33 +0000 (UTC) Received: from [98.139.91.64] by nm16.bullet.mail.sp2.yahoo.com with NNFMP; 27 Aug 2011 18:08:33 -0000 Received: from [98.139.91.32] by tm4.bullet.mail.sp2.yahoo.com with NNFMP; 27 Aug 2011 18:08:33 -0000 Received: from [127.0.0.1] by omp1032.mail.sp2.yahoo.com with NNFMP; 27 Aug 2011 18:08:33 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 175726.24219.bm@omp1032.mail.sp2.yahoo.com Received: (qmail 96456 invoked by uid 60001); 27 Aug 2011 18:08:32 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1314468512; bh=0Z0sCPHVGVDH4B686cIHcKF2fkYdyCdNnVhUUpoffW8=; h=X-YMail-OSG:Received:X-RocketYMMF:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=NjY/VOt76cnAIinLfFVbm5ahgyOp9DHAoCDXSbIfc70ruOpAuIpBpRW2JNvLWnkXPEeS8LApv0ggpoAyUsmvsJ+fkx8mow4nAj7TPmjMAV6ZntVVVz5KyzbGxcUeaXAgx7Z1za1XkysrVgoEg4zhcLmrIJH7WhQGFCRL+08wD1s= X-YMail-OSG: HlNkIPcVM1m6tYGV.bBL2os5IN5Gpib__gXeua2oO0UabbZ ZeBKa0sBdFiKaE1ZiAfagmgsn.YwB9B4lE8Bijnv_BLZ65BV4Exa3Brj7NJ0 7yYMGNQrnFzcDS7s09WFCa9EFvjLtbUEHFgjaRWKKAfyPufUEyanhKSDs3AZ 0wsK1qSqBl6G1yEtq_polDF7jnRbVgwjXKUzBgP_.Pgf7SMlXKj2Uv8pI3u8 PcU2UuZ59FCtJTlcAkUbE.JbZ8gLyNoMfK9tl25FIAJsqlT5DOJAPjM_RbN6 PxoMbwfWJR160C4d2Oo3c1yguc2K8BEoHho2LMOVhwSo8H5F3FqLhBWEztJW Z6uBzQDesfZuvuy9aLkgZcbVo3IErqxSv8xZC0MHAUZOO_DVkFRFNbVJRk_v ZW1ef4BraVWKeN4T61rFP7K0SFIDiOdmyx6D0Kusqup7OTm5dXGtjarnvxa_ Ups.yZnpkIodJyp3T.mr4yy9YgvT_jOdA7Iq6psACifzUw1naRe9XneVHD48 IyWvRaVqdJBbVK1x6 Received: from [200.118.157.7] by web113511.mail.gq1.yahoo.com via HTTP; Sat, 27 Aug 2011 11:08:32 PDT X-RocketYMMF: giffunip X-Mailer: YahooMailClassic/14.0.5 YahooMailWebService/0.8.113.315625 Message-ID: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> Date: Sat, 27 Aug 2011 11:08:32 -0700 (PDT) From: "Pedro F. Giffuni" To: freebsd-arch@freebsd.org In-Reply-To: <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: giffunip@tutopia.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 18:08:33 -0000 Hmm... I will admit quite frankly that the difference is not huge, and perhaps that's the reason why it hasn't been done before, but our bug reporting system is quite outdated. It works for most needs but I think the only reason we are stuck with it is because we can still file bugs from a console. I know this is not the first time someone brings the subject, we even have a Wiki page about it: http://wiki.freebsd.org/Bugzilla I do find Bugzilla a lot more useful than GNATS: I particularly like that it is possible to obsolete attachments. Requiring an id serves to control the spam a bit and develops some level of compromise with the project. In other projects (notably the Apache Foundation) JIRA is also gaining popularity. These newer bug tracking systems are very versatile and integrate with SVN. While here, some people have requested opengrok in the past. Just something to think, but delaying these changes too much rarely proves beneficial for the Project. Pedro. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 18:12:09 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8FA31065670 for ; Sat, 27 Aug 2011 18:12:09 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 73AC58FC16 for ; Sat, 27 Aug 2011 18:12:09 +0000 (UTC) Received: by qyk4 with SMTP id 4so1118779qyk.13 for ; Sat, 27 Aug 2011 11:12:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lN7cCHXqvkAbArUUnTcfxXiVds6TaAHkffnayZESLj0=; b=faM6MNE7dU2GWRd/vkSd4pQBpT7ACzeUeeLafc+s4YF5mcsxwA5FetRdSBMCb3PgAN boJ3YWZhrxjiNvg96VhUTrCkRsFSOQGwAv3zpk3bZ14/XIMW2NrN8eToa1QDsAwVlZ67 p3/c94rwoIxSA6fNvEg7KijmKJOzDupZxcdRM= MIME-Version: 1.0 Received: by 10.229.66.215 with SMTP id o23mr3505123qci.17.1314468728380; Sat, 27 Aug 2011 11:12:08 -0700 (PDT) Received: by 10.224.19.131 with HTTP; Sat, 27 Aug 2011 11:12:08 -0700 (PDT) In-Reply-To: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> References: <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> Date: Sat, 27 Aug 2011 11:12:08 -0700 Message-ID: From: Garrett Cooper To: giffunip@tutopia.com Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 18:12:09 -0000 On Sat, Aug 27, 2011 at 11:08 AM, Pedro F. Giffuni wrote: > Hmm... > > I will admit quite frankly that the difference is not huge, > and perhaps that's the reason why it hasn't been done > before, but our bug reporting system is quite outdated. > It works for most needs but I think the only reason > we are stuck with it is because we can still file bugs > from a console. > > I know this is not the first time someone brings the > subject, we even have a Wiki page about it: > > http://wiki.freebsd.org/Bugzilla > > I do find Bugzilla a lot more useful than GNATS: I > particularly like that it is possible to obsolete > attachments. Requiring an id serves to control the spam > a bit and develops some level of compromise with the > project. > > In other projects (notably the Apache Foundation) > JIRA is also gaining popularity. These newer bug > tracking systems are very versatile and integrate > with SVN. > > While here, some people have requested opengrok in > the past. > > Just something to think, but delaying these changes > too much rarely proves beneficial for the Project. There's also Trac as well, which has limited integration with some VCEs from what I've seen. Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 18:45:24 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DBE8106566B; Sat, 27 Aug 2011 18:45:24 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-7.mit.edu (DMZ-MAILSEC-SCANNER-7.MIT.EDU [18.7.68.36]) by mx1.freebsd.org (Postfix) with ESMTP id EDDB58FC12; Sat, 27 Aug 2011 18:45:23 +0000 (UTC) X-AuditID: 12074424-b7bcaae000000a05-1c-4e593b7262a2 Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) by dmz-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP id 01.33.02565.27B395E4; Sat, 27 Aug 2011 14:46:10 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id p7RIjNuS009832; Sat, 27 Aug 2011 14:45:23 -0400 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id p7RIjLvw011061 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 27 Aug 2011 14:45:22 -0400 (EDT) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id p7RIjKtF018726; Sat, 27 Aug 2011 14:45:20 -0400 (EDT) Date: Sat, 27 Aug 2011 14:45:20 -0400 (EDT) From: Benjamin Kaduk To: "Robert N. M. Watson" In-Reply-To: Message-ID: References: <4E53986B.5000804@FreeBSD.org> <201108230911.09021.jhb@freebsd.org> <4E564F15.3010809@FreeBSD.org> <201108250945.24606.jhb@freebsd.org> <4e576127.Suqlhieb0FMDx8cB%perryh@pluto.rain.com> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBIsWRmVeSWpSXmKPExsUixCmqrFtkHelnsPi0gMXs6dOYLPoOzWZ0 YPKY8Wk+SwBjFJdNSmpOZllqkb5dAldG58kvjAUrWCsO/7zD3sA4m6WLkZNDQsBE4u+W5YwQ tpjEhXvr2boYuTiEBPYxSlybtZEVwtnAKDF5+lMWCOcAk8TijReZIJwGRomZX7+xg/SzCGhL HJp0lBnEZhNQkZj5ZiMbiC0ioC/R338IqIaDg1lARmLOa2+QsLBAlMTX2WdZQMKcAvYSPw/n gYR5BRwkGnuXQS1+ziSx6fU6sJGiAjoSq/dPYYEoEpQ4OfMJmM0sYClx7s91tgmMgrOQpGYh SS1gZFrFKJuSW6Wbm5iZU5yarFucnJiXl1qka66Xm1mil5pSuokRFKDsLio7GJsPKR1iFOBg VOLhNZSM9BNiTSwrrsw9xCjJwaQkyttrDhTiS8pPqcxILM6ILyrNSS0+xCjBwawkwpuRE+En xJuSWFmVWpQPk5LmYFES57XZ6eAnJJCeWJKanZpakFoEk5Xh4FCS4FWxAhoqWJSanlqRlplT gpBm4uAEGc4DNHw6SA1vcUFibnFmOkT+FKOilDjvQpCEAEgiozQPrheWQF4xigO9Isw7G6SK B5h84LpfAQ1mAhqs4hgOMrgkESEl1cCokGh1//7UCxwVMzaac/4I/8agz+x6L+bqTsELR8uc S4RVroqU7W53vqmqPNOs9l0u/6WKvRyyy8+VlD8wrKv6tO7bZ9U4rphr/9f1vgvfd11d6dGD jJtT1W/1X9cp2Kxv/7qmoSVo+p1WO6ufgkVT8h2u2jgGJv8JLneVjRRePpUpV0PEXk2JpTgj 0VCLuag4EQCvGd0b+wIAAA== Cc: freebsd-arch@freebsd.org Subject: Re: Removing Giant from VFS in 10.0 (was: Re: skipping locks, mutex_owned, usb) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 18:45:24 -0000 On Sat, 27 Aug 2011, Robert N. M. Watson wrote: > > On 27 Aug 2011, at 02:43, Benjamin Kaduk wrote: > >> >> For what it's worth, OpenAFS already passes MNTK_MPSAFE /* solid steel */, with what I expect is nearly-correct locking. > > Excellent! > > (Although I guess OpenAFS internally has the moral equivalent of a Giant > lock that protects its own structures, but that's an entirely > independent problem that the OpenAFS community is already interested > in?) This is true. Though it may not actually be feasible to tackle until better integration with a lock-tracking framework such as WITNESS or Linux's (GPL-only) lock debugging framework. -Ben From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 19:07:03 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E84D106564A for ; Sat, 27 Aug 2011 19:07:03 +0000 (UTC) (envelope-from jlaffaye.freebsd@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 110828FC17 for ; Sat, 27 Aug 2011 19:07:02 +0000 (UTC) Received: by fxe4 with SMTP id 4so4354198fxe.13 for ; Sat, 27 Aug 2011 12:07:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=KU4kFpHfwUl9oOUhppxBM3FG1NSa2o3+zG3WLYxFUVw=; b=DP9TpmwUUwCuKtreW4yo9QXRrzSWdZZmmjR+iFx0+ika22Dnp/06jnOPEkq0rwhxAu TWUjJHVBt9adCRblpn6rtsDY52UML3vtcClKrW/VnCrjHFcrgQ8uwCLl2ybsyR3fxMMw 1JPec/KwlNsnp6VLe6c9YryvWHMQugBz8q1+E= Received: by 10.223.16.194 with SMTP id p2mr3873274faa.58.1314470196514; Sat, 27 Aug 2011 11:36:36 -0700 (PDT) Received: from chulak.jlaffaye.net (esc31-1-78-245-92-55.fbx.proxad.net [78.245.92.55]) by mx.google.com with ESMTPS id p11sm2334157faa.22.2011.08.27.11.36.35 (version=SSLv3 cipher=OTHER); Sat, 27 Aug 2011 11:36:35 -0700 (PDT) Sender: Julien Laffaye Message-ID: <4E593932.8060303@FreeBSD.org> Date: Sat, 27 Aug 2011 20:36:34 +0200 From: Julien Laffaye User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: giffunip@tutopia.com References: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> In-Reply-To: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 19:07:03 -0000 On 08/27/2011 20:08, Pedro F. Giffuni wrote: > Hmm... > > I will admit quite frankly that the difference is not huge, > and perhaps that's the reason why it hasn't been done > before, but our bug reporting system is quite outdated. > It works for most needs but I think the only reason > we are stuck with it is because we can still file bugs > from a console. > > I know this is not the first time someone brings the > subject, we even have a Wiki page about it: > > http://wiki.freebsd.org/Bugzilla We need someone to do the job. And it is not an "easy" migration in the sene that we need to keep old PRs, and if you take more time than you thought, everyone would yell at you (we need a bugtracker for releases management, ...). So, manpower, as usual :-) From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 19:14:08 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1FDB106564A for ; Sat, 27 Aug 2011 19:14:08 +0000 (UTC) (envelope-from giffunip@tutopia.com) Received: from nm25-vm2.bullet.mail.ne1.yahoo.com (nm25-vm2.bullet.mail.ne1.yahoo.com [98.138.91.213]) by mx1.freebsd.org (Postfix) with SMTP id 3CD628FC14 for ; Sat, 27 Aug 2011 19:14:07 +0000 (UTC) Received: from [98.138.90.50] by nm25.bullet.mail.ne1.yahoo.com with NNFMP; 27 Aug 2011 19:14:07 -0000 Received: from [98.138.89.192] by tm3.bullet.mail.ne1.yahoo.com with NNFMP; 27 Aug 2011 19:14:07 -0000 Received: from [127.0.0.1] by omp1050.mail.ne1.yahoo.com with NNFMP; 27 Aug 2011 19:14:07 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 651614.55983.bm@omp1050.mail.ne1.yahoo.com Received: (qmail 39409 invoked by uid 60001); 27 Aug 2011 19:14:07 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1314472447; bh=iTGj2bbav/5TSW+rYvA4aNXpbvlUJGLo477LGtGn4H4=; h=X-YMail-OSG:Received:X-RocketYMMF:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=1vMZM/7fmRkL1UeeYfHEpRzYG189TN9dXkId28YOxIyYYhwbIfGkUz39ku835+V4Jg3e1pZ3B57G40+PUwcynj9KShMASjdVWk/cRMoV3VMcF8FkyY2chyXb1SvnDzQOcaz72yrtqW23jfFKlYfZb2GgSUG8L6p2RnfvTg8OMFw= X-YMail-OSG: kR9is.0VM1mvKPDYhcG2Oh40shw2IOVQI2QWm8_D4oZ1QUw WH..q.grL4wj3wFXzzS0rLx6q10JzwbJN9VKIsMMUqqif1LZjXAYDqo.Pk4K FZNSuaP7aCvhtJ07lymUWeqF97M8kfzcenpclRNAZ5JRXi20Yp0vV9aYfz8o ZnIwBJgR764lRDAiwLUGOW5ZEPsuMithgN7Uw8jsuHG.fG4EwlZE2ozPBz47 gbnrE4XyjlrZZSt4F1SrtKIoBOF6fE_FqJ0OUCAJzX6nIuVfKf_586v5xwyP By1lz2IFDasy6tcYhSkT7PPCrI2ERvcisjjv_EcP13UAf99Fw70sVFPjDDHv FJyC.bv_K0v9Tq4ZZDwTYjex8MVyTC0CdFj4ZCJjHzKxACkqytmv3uOsiXRF z3_q5DonEVpLz.c8K4vxaX40rQjA__0h9kOR9sMrhmmddH7F2P8C_6b5Fc46 Q.s0- Received: from [200.118.157.7] by web113504.mail.gq1.yahoo.com via HTTP; Sat, 27 Aug 2011 12:14:06 PDT X-RocketYMMF: giffunip X-Mailer: YahooMailClassic/14.0.5 YahooMailWebService/0.8.113.315625 Message-ID: <1314472446.30399.YahooMailClassic@web113504.mail.gq1.yahoo.com> Date: Sat, 27 Aug 2011 12:14:06 -0700 (PDT) From: "Pedro F. Giffuni" To: Julien Laffaye In-Reply-To: <4E593932.8060303@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: giffunip@tutopia.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 19:14:08 -0000 --- On Sat, 8/27/11, Julien Laffaye wrote: ... > > > > I know this is not the first time someone brings the > > subject, we even have a Wiki page about it: > > > > http://wiki.freebsd.org/Bugzilla > > We need someone to do the job. And it is not an "easy" > migration in the sene that we need to keep old PRs, > and if you take more time than you thought, everyone > would yell at you (we need a bugtracker > for releases management, ...). > Yes, this is a consequence of GNATS being "obsolete": newer bug tracking systems have not kept the conversion tools updated as there are not many users for it anymore. We can always make drastic workarounds if the migration tools take too long or simply don't work. Here are two options: 1) Leave both systems running for a while: stop opening new PRs in GNATS: but leave it running for reference. Kill GNATS after two major releases. 2) Fix all bugs and make a perfect release (OK, not very realistic but theoretically possible ;-) ). > So, manpower, as usual :-) > Yes, and unfortunately the problem keeps growing :(. Pedro. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 19:21:38 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BBA31065670 for ; Sat, 27 Aug 2011 19:21:38 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id D02218FC13 for ; Sat, 27 Aug 2011 19:21:37 +0000 (UTC) Received: by vxh11 with SMTP id 11so4434432vxh.13 for ; Sat, 27 Aug 2011 12:21:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=m9iecy7tfrj8pNJgZCuJyLIFov8c3MXxOhDLcpL2Jw4=; b=A+2e2N/yVpPjcS1GXihdujZHTDzsN+gq7rdSyoMSLAwp3lxx2PJal3r6iRMtv7Zg9I Kpxi+DYaUv4bmFU3p+fpiUKFp6Rqq5vXfbFyilNgS88SZha6GFZOWkKQHK0DYgSuUXAJ X8NywegvBvcFAE3Pb5QvmlMFunUPNTELpw3mA= Received: by 10.52.65.240 with SMTP id a16mr2529781vdt.490.1314472897117; Sat, 27 Aug 2011 12:21:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.169.100 with HTTP; Sat, 27 Aug 2011 12:21:07 -0700 (PDT) In-Reply-To: <4E593932.8060303@FreeBSD.org> References: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> <4E593932.8060303@FreeBSD.org> From: Eitan Adler Date: Sat, 27 Aug 2011 15:21:07 -0400 Message-ID: To: Julien Laffaye Content-Type: text/plain; charset=UTF-8 Cc: giffunip@tutopia.com, freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 19:21:38 -0000 > We need someone to do the job. And it is not an "easy" migration in the sene > that we need to keep old PRs, and if you take more time than you thought, > everyone would yell at you (we need a bugtracker for releases management, > ...). Multiple people have stepped up and offered to help (including myself) once a proposal is accepted. Getting people to help for certain tasks is easier than one might think. As phk@ said we need to make this a group effort. > > So, manpower, as usual :-) > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > -- Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 19:29:41 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2885106566B for ; Sat, 27 Aug 2011 19:29:41 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id AB7E78FC0A for ; Sat, 27 Aug 2011 19:29:41 +0000 (UTC) Received: by vxh11 with SMTP id 11so4437638vxh.13 for ; Sat, 27 Aug 2011 12:29:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=BA9akaGU1GS2BorApO8dVFrs8o3IZcgZV+sKnvy7Mzs=; b=oDX/IdTBujZZhkUEhEyMTz9VynisMiMtgigqDqk+1L9kFyABzwWtD8Je72P+g8Qr+P Q356depp55bv5egSYw8KgDGuYxRTfq92ogD8VX8FAIyFza7oWVJVViJB/+WCBGzKdPti Yl6/RnJRgGmaegf0cNHUVcCnVGC32qRfz0KzE= Received: by 10.52.95.166 with SMTP id dl6mr2665887vdb.88.1314471633203; Sat, 27 Aug 2011 12:00:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.169.100 with HTTP; Sat, 27 Aug 2011 12:00:03 -0700 (PDT) In-Reply-To: References: <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> From: Eitan Adler Date: Sat, 27 Aug 2011 15:00:03 -0400 Message-ID: To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Cc: giffunip@tutopia.com, freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 19:29:42 -0000 [removing specific proposals here to make a general comment] I don't think anyone wants to keep GNATS. However most proposals of this type are "lets use bug tracking system X because it has features A, B, C." With every tool change comes a workflow change and we need to be able to deal with that. As a member of the bugbusting team one of the most useful features for me has been the ability to use shell scripts to change bug states. As a port maintainer one of the most useful features has been the ability to use send-pr from the command line _without registration_. As a comitter having my commits automatically posted to the PR was very convenient. I am obviously not the only user and what is important to me might be trivial to other people. If one wants to gain the support of the community, and to have a good proposal we need to work forwards: What are the requirements we are looking for in a bug tracking system? Only then could we look to see what software fulfills this. In order to achieve that goal I am working on two things: a) A wiki page which we could use to describe the features we want *in order of priority* and b) A Google form with a list of features and related issues to act as a survey. The former is quite empty at the moment. The latter will be announced (hopefully) shortly -- Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 19:30:00 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B397106566C for ; Sat, 27 Aug 2011 19:30:00 +0000 (UTC) (envelope-from giffunip@tutopia.com) Received: from nm19-vm1.bullet.mail.ne1.yahoo.com (nm19-vm1.bullet.mail.ne1.yahoo.com [98.138.91.56]) by mx1.freebsd.org (Postfix) with SMTP id 9102B8FC17 for ; Sat, 27 Aug 2011 19:29:59 +0000 (UTC) Received: from [98.138.90.53] by nm19.bullet.mail.ne1.yahoo.com with NNFMP; 27 Aug 2011 19:29:59 -0000 Received: from [98.138.88.234] by tm6.bullet.mail.ne1.yahoo.com with NNFMP; 27 Aug 2011 19:29:59 -0000 Received: from [127.0.0.1] by omp1034.mail.ne1.yahoo.com with NNFMP; 27 Aug 2011 19:29:59 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 4666.15469.bm@omp1034.mail.ne1.yahoo.com Received: (qmail 48576 invoked by uid 60001); 27 Aug 2011 19:29:58 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1314473398; bh=Cm6r3Bm6oAR9EntXA0JmQ4btRn+EUpJ4MNb83mK4oak=; h=X-YMail-OSG:Received:X-RocketYMMF:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=YpK2cqC4lLz2vhtmhF08WFGJZ+5Lx2Kgw4YqZMFaQkpQuYtKjCp2cNLge9PWEY1i1JP5dyKLBFHrkolW73mGYxkgXD6MxtHDND+YGB+Muq7pj14HY+Kv5la3ZzQkvpik8TeyuZd+JMqL3te5GAVILhL4WWTBn32nvNQYWCHNE3Y= X-YMail-OSG: EZSi00sVM1kf2GED6VtGBPgrtRfD32HTu50xszrb3KTiXHI h4vlqXwJU164psLTvxqntv_6GShK5LQ7t0mxK9m0eUfmhfhFDqs5Uip9pp.b 6SJgENvPWVFyUCZ3Q7RIHsX3wznKODuT4nWhcP7BHWK1GvK4eAxsHnmV358a LpVHd2xGkgeuyyF6QR1yMcWOc4yHKDCfJKlHSqNrLqpfIzYqDKZKJDrf8sLR FnJbC47IrFsX8.tFANg0uhrUihL.G6IOBLlhFFfOSr7Tnl5_m_k.QeKscaL_ n0Jk.jmBVfCX8qZwmID1tp8J1XBIT..zhJbm.PJsVi6RnlfMUTsuDs1zpdhG BTj6amBtqoHcjvdRNnF4tC2Q4DifbmWQMZ1_2Uv7UYsJhEZIeIjhXUqldAAK uvrH8J1qf_PoLWBXMyiBpElbfK1FfqL7kGJgLhKAvTCFlc6ULxiNEzvD3cbO upbU- Received: from [200.118.157.7] by web113509.mail.gq1.yahoo.com via HTTP; Sat, 27 Aug 2011 12:29:58 PDT X-RocketYMMF: giffunip X-Mailer: YahooMailClassic/14.0.5 YahooMailWebService/0.8.113.315625 Message-ID: <1314473398.47547.YahooMailClassic@web113509.mail.gq1.yahoo.com> Date: Sat, 27 Aug 2011 12:29:58 -0700 (PDT) From: "Pedro F. Giffuni" To: Eitan Adler In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: giffunip@tutopia.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 19:30:00 -0000 Hi; --- On Sat, 8/27/11, Eitan Adler wrote: ... I agree it's difficult to determine exactly which tool to use. > > With every tool change comes a workflow change and we need > to be able to deal with that. As a member of the bugbusting > team one of the most useful features for me has been the > ability to use shell scripts to change bug states. As a > port maintainer one of the most useful features has been > the ability to use send-pr from the command line > _without registration_. I do think port maintainers should have some form of ID in the bug reporting system. Casual users probably won't send a PR anyways but instead will contact the port maintainer first. ... (I snipped without mercy a lot of good stuff here) > > In order to achieve that goal I am working on two things: > a) A wiki page which we could use to describe the features > we want *in order of priority* and b) A Google form with > a list of features and related issues to act as a survey. > The former is quite empty at the moment. > The latter will be announced (hopefully) shortly > Surveys are good but you have to tie it to the specific alternatives, otherwise you will reach the already known answer that a perfect bug reporting system doesn't exist. Pedro. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 19:49:39 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3FBA1065672 for ; Sat, 27 Aug 2011 19:49:39 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-7.mit.edu (DMZ-MAILSEC-SCANNER-7.MIT.EDU [18.7.68.36]) by mx1.freebsd.org (Postfix) with ESMTP id 74A3B8FC16 for ; Sat, 27 Aug 2011 19:49:39 +0000 (UTC) X-AuditID: 12074424-b7bcaae000000a05-89-4e594a81a568 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) by dmz-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP id 31.54.02565.18A495E4; Sat, 27 Aug 2011 15:50:25 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id p7RJncVp010208 for ; Sat, 27 Aug 2011 15:49:38 -0400 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id p7RJnZMa016581 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sat, 27 Aug 2011 15:49:38 -0400 (EDT) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id p7RJnZ2W019255; Sat, 27 Aug 2011 15:49:35 -0400 (EDT) Date: Sat, 27 Aug 2011 15:49:35 -0400 (EDT) From: Benjamin Kaduk To: freebsd-arch@freebsd.org In-Reply-To: Message-ID: References: <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrBIsWRmVeSWpSXmKPExsUixCmqrdvoFelnMHWCiMXs6dOYHBg9Znya zxLAGMVlk5Kak1mWWqRvl8CVMW35b7aCz2wVx+78Y2lg3MHaxcjJISFgIjHr8yE2CFtM4sK9 9UA2F4eQwD5Gifvv29khnPOMEn2rmqAy95kkZsz6A+U0MEr8uT4HrJ9FQFti1rQNLCA2m4CK xMw3G8HiIgIyEmvmzADbJyyQJXGzfxVQnIODUyBQomEnWDmvgIPEpmMfWCFmXmaUuPxqC1hC VEBHYvX+KVBFghInZz4Bs5kFLCXO/bnONoFRYBaS1CwkqQWMTKsYZVNyq3RzEzNzilOTdYuT E/PyUot0zfVyM0v0UlNKNzGCApDdRWUHY/MhpUOMAhyMSjy8hpKRfkKsiWXFlbmHGCU5mJRE eRM9gUJ8SfkplRmJxRnxRaU5qcWHGCU4mJVEeDNyIvyEeFMSK6tSi/JhUtIcLErivDY7HfyE BNITS1KzU1MLUotgsjIcHEoSvPdBhgoWpaanVqRl5pQgpJk4OEGG8wANvwRSw1tckJhbnJkO kT/FqCglznsPJCEAksgozYPrhSWIV4ziQK8I8yaDVPEAkwtc9yugwUxAg1Ucw0EGlyQipKQa GBNYnX37Fm+//nLrUUkvx5g5HYzzz336PaX/5/G+09yfP8U6P2/zLvnTc6OgdHvHj0+ep49d Ltab5r5EUesVo+Bn/dcM9rHtT69LWmWxb0l9/d2bZ4foZJ3HtxV8DAIVPp4RfG5zafrLW6v6 G2Vq7m3+WFKUzbjrncPHA09N7se0hWsXVn656qjEUpyRaKjFXFScCADGg+Eb6wIAAA== Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 19:49:39 -0000 We have used JIRA, Trac, and Request Tracker for various things here at MIT. We phased out JIRA because it was a hassle to work with. Trac is pretty slow in our current deployment, and it may or may not get better once we move to different hardware (I personally don't think it will be great). RT is I think more for user support tickets than proper bug tracking (though we use it for bug tracking at OpenAFS); I don't think it has quite the workflow we want, and I gather it can be pretty complicated to mention. I will also mention the Debian bug tracker which, despite the jokes made at its expense in various Linux communities, seems like it may be a reasonable fit for what we want. Most interaction is done over email, but I could imagine writing scripts around it. Sadly, I don't have any real experience with Bugzilla for comparison. -Ben Kaduk From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 20:02:51 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF626106566B for ; Sat, 27 Aug 2011 20:02:51 +0000 (UTC) (envelope-from ohauer@FreeBSD.org) Received: from p578be941.dip0.t-ipconnect.de (p578be941.dip0.t-ipconnect.de [87.139.233.65]) by mx1.freebsd.org (Postfix) with ESMTP id 7C15F8FC14 for ; Sat, 27 Aug 2011 20:02:51 +0000 (UTC) Received: from [192.168.0.100] (cde1100.uni.vrs [192.168.0.100]) (Authenticated sender: ohauer) by p578be941.dip0.t-ipconnect.de (Postfix) with ESMTPSA id F006320818; Sat, 27 Aug 2011 22:02:46 +0200 (CEST) Message-ID: <4E594D6A.9060407@FreeBSD.org> Date: Sat, 27 Aug 2011 22:02:50 +0200 From: Olli Hauer User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: giffunip@tutopia.com References: <1314473398.47547.YahooMailClassic@web113509.mail.gq1.yahoo.com> In-Reply-To: <1314473398.47547.YahooMailClassic@web113509.mail.gq1.yahoo.com> X-Enigmail-Version: 1.3.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Eitan Adler , freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ohauer@FreeBSD.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 20:02:51 -0000 On 2011-08-27 21:29, Pedro F. Giffuni wrote: > Hi; > > --- On Sat, 8/27/11, Eitan Adler wrote: > > ... > > I agree it's difficult to determine exactly which tool to > use. > >> >> With every tool change comes a workflow change and we need >> to be able to deal with that. As a member of the bugbusting >> team one of the most useful features for me has been the >> ability to use shell scripts to change bug states. As a >> port maintainer one of the most useful features has been >> the ability to use send-pr from the command line >> _without registration_. I'm working with both gnats (on FreeBSD) and bugzilla / Jira on work. Both systems have the advantages / disadvantages. Bugzilla with A good customized UI, a well designed workflow will be an improvement for users, devs, port committers/maintainers. I don't know how many devs / port committers are working with a csup'd gnats database but this and the command line usage is a real advantage for me. [ snipp ... ] From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 20:11:43 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7DAB2106564A for ; Sat, 27 Aug 2011 20:11:43 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 267758FC13 for ; Sat, 27 Aug 2011 20:11:43 +0000 (UTC) Received: by qyk9 with SMTP id 9so3423481qyk.13 for ; Sat, 27 Aug 2011 13:11:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wzEFJ/rbNcYLsY+4n1dKUTj6BH50mkXbqApctHEwTa8=; b=kj3PUL/yIiI1nsMlx9+XcVii95CJXQb/5Lj4dTgQyY1gdRSe1FjsXSB2Jzf2qNC2xP zuCzTIMvcknkUhwl/GP0dpR2ZDwpPZCKdphRhEGM5e6cx3DM7NE3UrbUUB0HhCh0lE6m z0wQOrapaX/HO+THpCzJtJpVgZJEWOnzjf6fc= MIME-Version: 1.0 Received: by 10.229.66.215 with SMTP id o23mr3627826qci.17.1314475902585; Sat, 27 Aug 2011 13:11:42 -0700 (PDT) Received: by 10.224.19.131 with HTTP; Sat, 27 Aug 2011 13:11:42 -0700 (PDT) In-Reply-To: <4E594D6A.9060407@FreeBSD.org> References: <1314473398.47547.YahooMailClassic@web113509.mail.gq1.yahoo.com> <4E594D6A.9060407@FreeBSD.org> Date: Sat, 27 Aug 2011 13:11:42 -0700 Message-ID: From: Garrett Cooper To: ohauer@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Eitan Adler , giffunip@tutopia.com, freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 20:11:43 -0000 On Sat, Aug 27, 2011 at 1:02 PM, Olli Hauer wrote: > On 2011-08-27 21:29, Pedro F. Giffuni wrote: >> Hi; >> >> --- On Sat, 8/27/11, Eitan Adler wrote: >> >> ... >> >> I agree it's difficult to determine exactly which tool to >> use. >> >>> >>> With every tool change comes a workflow change and we need >>> to be able to deal with that. As a member of the bugbusting >>> team one of the most useful features for me has been the >>> ability to use shell scripts to change bug states. As a >>> port maintainer one of the most useful features has been >>> the ability to use send-pr from the command line >>> _without registration_. > > I'm working with both gnats (on FreeBSD) and bugzilla / Jira on work. > Both systems have the advantages / disadvantages. > Bugzilla with A good customized UI, a well designed workflow > will be an improvement for users, devs, port committers/maintainers. > > I don't know how many devs / port committers are working with a > csup'd gnats database but this and the command line usage is a real > advantage for me. CLI access for GNATS can be achieved via Bugzilla (some SQL fu involved IIRC), but in all honesty turning off the anonymous submission functionality might make some of the bugbusters lives easier (I remember several times that spammers tried pounding the crud out of GNATS with useless requests). Registration/authentication is a model that other projects adopt -- why not just do things this way? Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 20:28:03 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7BFB9106564A for ; Sat, 27 Aug 2011 20:28:03 +0000 (UTC) (envelope-from jos@catnook.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 04E438FC0C for ; Sat, 27 Aug 2011 20:28:02 +0000 (UTC) Received: by vws18 with SMTP id 18so4900828vws.13 for ; Sat, 27 Aug 2011 13:28:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.21.205 with SMTP id x13mr2631202vde.515.1314475331440; Sat, 27 Aug 2011 13:02:11 -0700 (PDT) Received: by 10.52.113.199 with HTTP; Sat, 27 Aug 2011 13:02:11 -0700 (PDT) Received: by 10.52.113.199 with HTTP; Sat, 27 Aug 2011 13:02:11 -0700 (PDT) In-Reply-To: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> References: <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> Date: Sat, 27 Aug 2011 13:02:11 -0700 Message-ID: From: Jos Backus To: giffunip@tutopia.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 20:28:03 -0000 What about Redmine? It supports Subversion and Git, among other things. http://www.redmine.org/ Jos On Aug 27, 2011 11:08 AM, "Pedro F. Giffuni" wrote: > Hmm... > > I will admit quite frankly that the difference is not huge, > and perhaps that's the reason why it hasn't been done > before, but our bug reporting system is quite outdated. > It works for most needs but I think the only reason > we are stuck with it is because we can still file bugs > from a console. > > I know this is not the first time someone brings the > subject, we even have a Wiki page about it: > > http://wiki.freebsd.org/Bugzilla > > I do find Bugzilla a lot more useful than GNATS: I > particularly like that it is possible to obsolete > attachments. Requiring an id serves to control the spam > a bit and develops some level of compromise with the > project. > > In other projects (notably the Apache Foundation) > JIRA is also gaining popularity. These newer bug > tracking systems are very versatile and integrate > with SVN. > > While here, some people have requested opengrok in > the past. > > Just something to think, but delaying these changes > too much rarely proves beneficial for the Project. > > Pedro. > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 20:41:50 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77FE3106566B for ; Sat, 27 Aug 2011 20:41:50 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 389A58FC0A for ; Sat, 27 Aug 2011 20:41:49 +0000 (UTC) Received: by qyk9 with SMTP id 9so3430113qyk.13 for ; Sat, 27 Aug 2011 13:41:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Fljwb0FZZ2MkOseml7ypB8QVbYutM9OMFq+cJzQgtOk=; b=ntgx3DMLQM0Y+e5pEUWQXNEPY4t9uWXH5At1kXBmisrtAI92vkF9jwbDFTDHoRB8+2 p5RqUfRMHAogaX2PRsAK9YgOmu4wujZc6IX+4qLXrrzbuUgiTmrvHDP+DbJmiNMvc7sj OOBTBCAAC2byJcqVhn0IlK98tntnvtmZK3Yrs= MIME-Version: 1.0 Received: by 10.229.229.17 with SMTP id jg17mr2880814qcb.131.1314477709519; Sat, 27 Aug 2011 13:41:49 -0700 (PDT) Received: by 10.224.19.131 with HTTP; Sat, 27 Aug 2011 13:41:49 -0700 (PDT) In-Reply-To: References: <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> Date: Sat, 27 Aug 2011 13:41:49 -0700 Message-ID: From: Garrett Cooper To: Jos Backus Content-Type: text/plain; charset=ISO-8859-1 Cc: giffunip@tutopia.com, freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 20:41:50 -0000 On Sat, Aug 27, 2011 at 1:02 PM, Jos Backus wrote: > What about Redmine? It supports Subversion and Git, among other things. > > http://www.redmine.org/ It uses rails though. Some of my friends who have maintained rails installs say that maintenance is a nightmare because it's ruby based. -Garrett From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 20:43:22 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B827106564A for ; Sat, 27 Aug 2011 20:43:22 +0000 (UTC) (envelope-from giffunip@tutopia.com) Received: from nm28.bullet.mail.sp2.yahoo.com (nm28.bullet.mail.sp2.yahoo.com [98.139.91.98]) by mx1.freebsd.org (Postfix) with SMTP id 26A148FC0C for ; Sat, 27 Aug 2011 20:43:21 +0000 (UTC) Received: from [98.139.91.69] by nm28.bullet.mail.sp2.yahoo.com with NNFMP; 27 Aug 2011 20:43:21 -0000 Received: from [98.139.91.21] by tm9.bullet.mail.sp2.yahoo.com with NNFMP; 27 Aug 2011 20:43:21 -0000 Received: from [127.0.0.1] by omp1021.mail.sp2.yahoo.com with NNFMP; 27 Aug 2011 20:43:21 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 692751.44397.bm@omp1021.mail.sp2.yahoo.com Received: (qmail 38188 invoked by uid 60001); 27 Aug 2011 20:43:21 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1314477801; bh=UI85avryGyKJKZ7pT97+6Zsj/E7oSE8MG/7LCavxAyQ=; h=X-YMail-OSG:Received:X-RocketYMMF:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:Cc:MIME-Version:Content-Type; b=hfylod/C/1NMgyFyl1L/YLgAS81iXhfWiyfJ72vuK/h53CgrtfQ/dmubS3uweNqEo/7XrEJfNGU4uoP4Dg6Dvb66o0MsPmqOBCas/TDTf7bDQpJwT4GrrAalTVoDqBpbUvod0mHkWrPwzmk93TFdvDuU7IRkKFL8RlXLs3xotgA= X-YMail-OSG: iRS3JIsVM1kl_KaUwQfr7UoYmxxXOdSJtHxzKPx2R8XWynR 5ScnPCqaUSPoEK7mtCTyRUuQtXMCmsrLU_wm5JBKkof7cWVuET2RGoJql7QU OIXee1VZJc8Z2y8GHAl_kh9HDkQLFSBVKwkX3PR7niZTaUEWFJn6nMl.Xz_t T79HonWAeZuiM_7AriVyVrTtrj1BBu5N83kH98zYZZv_DQoqRscpqiKASgDu KbOCAn.ZFAkBGaljpZs2VdrLqpzq5h410FT6SrHrqzfHsL8q.mJDpJxdF740 t_xjvMaNh_BLBH4d5cBXKBKGmhHy4A67EHdb6KNrEzqgfEPpwSpr3.WLSbf3 A1atHGhb.K4qRw_cS8J0V924E5IhWYHmIb7uFZknH8F77ugS9xl8shk6S6Ng 0leVdZ3oA6AV2h6DmkrtAkXlGGXKEVEwwtYZttO6O0eJYYXOySQFzpLA.R1h L0tf1avMAlLTzt6vTDnurxAEqjap65gsfRjhKswNzxGGTInDwYnUqVIiKPrv ZOwH9 Received: from [200.118.157.7] by web113508.mail.gq1.yahoo.com via HTTP; Sat, 27 Aug 2011 13:43:21 PDT X-RocketYMMF: giffunip X-Mailer: YahooMailClassic/14.0.5 YahooMailWebService/0.8.113.315625 Message-ID: <1314477801.33113.YahooMailClassic@web113508.mail.gq1.yahoo.com> Date: Sat, 27 Aug 2011 13:43:21 -0700 (PDT) From: "Pedro F. Giffuni" To: Jos Backus MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: giffunip@tutopia.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 20:43:22 -0000 Hi; --- On Sat, 8/27/11, Jos Backus wrote: > What about Redmine? It supports Subversion and Git, > among other things. > http://www.redmine.org/ > Jos It looks nice, however I couldn't find information about migration from GNATS. It looks like the safest path for conversion from GNATS to *anything* passes through bugzilla. Maybe I haven't looked thoroughly but there's also a conversion tool for Mantis, and another option is generating .csv files. cheers, Pedro. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 20:49:06 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59B08106566B for ; Sat, 27 Aug 2011 20:49:06 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 19EA68FC13 for ; Sat, 27 Aug 2011 20:49:05 +0000 (UTC) Received: by qwc9 with SMTP id 9so3373047qwc.13 for ; Sat, 27 Aug 2011 13:49:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Swn50pXCLnH5TUUBAmy8e/JphiDox/xHP3VwgXjoyf8=; b=KHvTo8ZjZUP3vTrCW2NM3umzJRMdAYUF7Qoh2s39/lvFgTXzkQGJHeycSWXKC33a0d hd0vOMSIKBt9dDo/2nA2cg2qyg3t4e8kN73elnwq6m+DRrD9D4PIA3DT+Y4lSj9a3MRJ /skYqtTxcyjaJNO0NHyKLUUUa8JbraxbItqjc= MIME-Version: 1.0 Received: by 10.229.229.17 with SMTP id jg17mr2886864qcb.131.1314478145356; Sat, 27 Aug 2011 13:49:05 -0700 (PDT) Received: by 10.224.19.131 with HTTP; Sat, 27 Aug 2011 13:49:05 -0700 (PDT) In-Reply-To: <1314477801.33113.YahooMailClassic@web113508.mail.gq1.yahoo.com> References: <1314477801.33113.YahooMailClassic@web113508.mail.gq1.yahoo.com> Date: Sat, 27 Aug 2011 13:49:05 -0700 Message-ID: From: Garrett Cooper To: giffunip@tutopia.com Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 20:49:06 -0000 On Sat, Aug 27, 2011 at 1:43 PM, Pedro F. Giffuni wrote: > Hi; > > --- On Sat, 8/27/11, Jos Backus wrote: > >> What about Redmine? It supports Subversion and Git, >> among other things. >> http://www.redmine.org/ >> Jos > > It looks nice, however I couldn't find information about > migration from GNATS. > > It looks like the safest path for conversion from GNATS > to *anything* passes through bugzilla. > > Maybe I haven't looked thoroughly but there's also a > conversion tool for Mantis, and another option is > generating .csv files. Given a programmatic API or CLI method for importing records into the new system and access for testing, it wouldn't be all that hard for me to implement an importer given a couple of weekend's time. Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Sat Aug 27 21:57:25 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B938C106566C for ; Sat, 27 Aug 2011 21:57:25 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 494BF8FC14 for ; Sat, 27 Aug 2011 21:57:24 +0000 (UTC) Received: from mart.js.berklix.net (p5DCBF61D.dip.t-dialin.net [93.203.246.29]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p7RLvMiT087527; Sat, 27 Aug 2011 21:57:23 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id p7RLvBPu000260; Sat, 27 Aug 2011 23:57:11 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.4/8.14.4) with ESMTP id p7RLuwK5047179; Sat, 27 Aug 2011 21:57:05 GMT (envelope-from jhs@fire.js.berklix.net) Message-Id: <201108272157.p7RLuwK5047179@fire.js.berklix.net> To: Benjamin Kaduk From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Sat, 27 Aug 2011 15:49:35 EDT." Date: Sat, 27 Aug 2011 23:56:58 +0200 Sender: jhs@berklix.com Cc: freebsd-arch@freebsd.org Subject: Re: How about finally replacing GNATS? (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2011 21:57:25 -0000 Benjamin Kaduk wrote: > We have used JIRA, Trac, and Request Tracker ... Might there be another newish/ interesting bug tracker here ? http://gforge.cs.vu.nl/gf/project/minix/tracker/ >From http://www.minix3.org Whois minix3.org shows 22-Jun-2005, PS I recall the original restrictive Prentice Hall copyright at & before Minix 1.3 ~ 1991, Now Minix has a BSD style license, & Device drivers run as user processes ! 301439387 minix_R3.1.8-r8398.iso.bz2 72 M minix3_2_0_ide_20110722_1e56737131.iso.bz2 Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Reply below, not above; Indent with "> "; Cumulative like a play script. Format: Plain text. Not HTML, multipart/alternative, base64, quoted-printable.