From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 18:15:47 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0882F6EB for ; Sun, 22 Sep 2013 18:15:47 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DB0CA2E5C for ; Sun, 22 Sep 2013 18:15:46 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id r8MIFecR008816 for freebsd-hackers@freebsd.org; Sun, 22 Sep 2013 18:15:40 GMT (envelope-from kientzle@freebsd.org) Received: from [192.168.2.123] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id vvbeyjfzuxen78rwyrkgke7sns; for freebsd-hackers@freebsd.org; Sun, 22 Sep 2013 18:15:40 +0000 (UTC) (envelope-from kientzle@freebsd.org) From: Tim Kientzle Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Tracking -CURRENT with VMWare Fusion and Crochet Message-Id: <23B55777-C334-46BE-86ED-522EDF228254@freebsd.org> Date: Sun, 22 Sep 2013 11:15:40 -0700 To: FreeBSD Hackers Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) X-Mailer: Apple Mail (2.1510) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 18:15:47 -0000 I'm experimenting with a new way to track -CURRENT by using Crochet to build successive VMWare VM images. Each new VM can then be used to build the next one. I'm specifically using this with VMWare Fusion on Mac OS, but it should work with Linux or FreeBSD VMWare hosts as well. Here's an outline of how to set this up: = https://github.com/kientzle/crochet-freebsd/wiki/Tracking-FreeBSD-CURRENT-= with-VMWare-Fusion-and-Crochet There are still a few issues I need to work out before I'm entirely happy with this approach, but it's looking pretty promising so far. Feedback appreciated. Cheers, Tim From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 19:59:44 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1D1CDC67; Sun, 22 Sep 2013 19:59:44 +0000 (UTC) (envelope-from melifaro@yandex-team.ru) Received: from forward-corp1g.mail.yandex.net (forward-corp1g.mail.yandex.net [IPv6:2a02:6b8:0:1402::10]) by mx1.freebsd.org (Postfix) with ESMTP id 69E7022F4; Sun, 22 Sep 2013 19:59:43 +0000 (UTC) Received: from smtpcorp4.mail.yandex.net (smtpcorp4.mail.yandex.net [95.108.252.2]) by forward-corp1g.mail.yandex.net (Yandex) with ESMTP id DD945366009A; Sun, 22 Sep 2013 23:59:40 +0400 (MSK) Received: from smtpcorp4.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp4.mail.yandex.net (Yandex) with ESMTP id 1E1F82C0058; Sun, 22 Sep 2013 23:59:40 +0400 (MSK) Received: from dhcp170-36-red.yandex.net (dhcp170-36-red.yandex.net [95.108.170.36]) by smtpcorp4.mail.yandex.net (nwsmtp/Yandex) with ESMTP id vYfnD1rFej-xei4WIF6; Sun, 22 Sep 2013 23:59:40 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1379879980; bh=PTdCNocghNrGKUJpHQ2f83+w9MCRgnEAdFhS+RlHncU=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=m6k8ItAvMBHUq9A03HVz2UgoLshfWFr7lzQILEivApjPY3CedK7sTAjtsEYuN6dK2 cqo8EiYKcUUqHgbvGbjhOwqPb2rZmcIQEpWaRpLv+OQdJsfpFLlXSuBZOmJWZ67rA6 lbBAKa1YMckWMMC+v7GnojZuQt6OwTrLUKzTw3+o= Authentication-Results: smtpcorp4.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <523F4BED.6000804@yandex-team.ru> Date: Sun, 22 Sep 2013 23:58:37 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130824 Thunderbird/17.0.8 MIME-Version: 1.0 To: Andre Oppermann Subject: Re: Network stack changes References: <521E41CB.30700@yandex-team.ru> <521E78B0.6080709@freebsd.org> In-Reply-To: <521E78B0.6080709@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 22 Sep 2013 22:24:38 +0000 Cc: adrian@freebsd.org, freebsd-hackers@freebsd.org, FreeBSD Net , luigi@freebsd.org, ae@FreeBSD.org, Gleb Smirnoff , freebsd-arch@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 19:59:44 -0000 On 29.08.2013 02:24, Andre Oppermann wrote: > On 28.08.2013 20:30, Alexander V. Chernikov wrote: >> Hello list! > > Hello Alexander, Hello Andre! I'm very sorry to answer so late. > > you sent quite a few things in the same email. I'll try to respond > as much as I can right now. Later you should split it up to have > more in-depth discussions on the individual parts. > > If you could make it to the EuroBSDcon 2013 DevSummit that would be > even more awesome. Most of the active network stack people will be > there too. I've sent presentation describing nearly the same things to devsummit@ so I hope this can be discussed in Networking group. I hope to attend DevSummit & EuroBSDcon. > >> There is a lot constantly raising discussions related to networking >> stack performance/changes. >> >> I'll try to summarize current problems and possible solutions from my >> point of view. >> (Generally this is one problem: stack is >> slooooooooooooooooooooooooooow, but we need to know why and >> what to do). > > Compared to others its not thaaaaaaat slow. ;) > >> Let's start with current IPv4 packet flow on a typical router: >> http://static.ipfw.ru/images/freebsd_ipv4_flow.png >> >> (I'm sorry I can't provide this as text since Visio don't have any >> 'ascii-art' exporter). >> >> Note that we are using process-to-completion model, e.g. process any >> packet in ISR until it is either >> consumed by L4+ stack or dropped or put to egress NIC queue. >> >> (There is also deferred ISR model implemented inside netisr but it >> does not change much: >> it can help to do more fine-grained hashing (for GRE or other similar >> traffic), but >> 1) it uses per-packet mutex locking which kills all performance >> 2) it currently does not have _any_ hashing functions (see absence of >> flags in `netstat -Q`) >> People using http://static.ipfw.ru/patches/netisr_ip_flowid.diff (or >> modified PPPoe/GRE version) >> report some profit, but without fixing (1) it can't help much >> ) >> >> So, let's start: >> >> 1) Ixgbe uses mutex to protect each RX ring which is perfectly fine >> since there is nearly no contention >> (the only thing that can happen is driver reconfiguration which is >> rare and, more signifficant, we >> do this once >> for the batch of packets received in given interrupt). However, due >> to some (im)possible deadlocks >> current code >> does per-packet ring unlock/lock (see ixgbe_rx_input()). >> There was a discussion ended with nothing: >> http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033520.html >> >> 1*) Possible BPF users. Here we have one rlock if there are any >> readers present >> (and mutex for any matching packets, but this is more or less OK. >> Additionally, there is WIP to >> implement multiqueue BPF >> and there is chance that we can reduce lock contention there). > > Rlock to rmlock? Yes, probably. > >> There is also an "optimize_writers" hack permitting applications >> like CDP to use BPF as writers but not registering them as receivers >> (which implies rlock) > > I believe longer term we should solve this with a protocol type > "ethernet" > so that one can send/receive ethernet frames through a normal socket. Yes. AF_LINK or any similar. > >> 2/3) Virtual interfaces (laggs/vlans over lagg and other simular >> constructions). >> Currently we simply use rlock to make s/ix0/lagg0/ and, what is much >> more funny - we use complex >> vlan_hash with another rlock to >> get vlan interface from underlying one. >> >> This is definitely not like things should be done and this can be >> changed more or less easily. > > Indeed. > >> There are some useful terms/techniques in world of software/hardware >> routing: they have clear >> 'control plane' and 'data plane' separation. >> Former one is for dealing control traffic (IGP, MLD, IGMP snooping, >> lagg hellos, ARP/NDP, etc..) and >> some data traffic (packets with TTL=1, with options, destined to >> hosts without ARP/NDP record, and >> similar). Latter one is done in hardware (or effective software >> implementation). >> Control plane is responsible to provide data for efficient data plane >> operations. This is the point >> we are missing nearly everywhere. > > ACK. > >> What I want to say is: lagg is pure control-plane stuff and vlan is >> nearly the same. We can't apply >> this approach to complex cases like >> lagg-over-vlans-over-vlans-over-(pppoe_ng0-and_wifi0) >> but we definitely can do this for most common setups like (igb* or >> ix* in lagg with or without vlans >> on top of lagg). > > ACK. > >> We already have some capabilities like VLANHWFILTER/VLANHWTAG, we can >> add some more. We even have >> per-driver hooks to program HW filtering. > > We could. Though for vlan it looks like it would be easier to remove the > hardware vlan tag stripping and insertion. It only adds complexity in > all > drivers for no gain. No. Actually as far as I understand it helps driver to perform TSO. Anyway, IMO we should use HW capabilities if we can. (this probably does not add much speed on 1G, but on 10/20/40G this can help much more). > >> One small step to do is to throw packet to vlan interface directly >> (P1), proof-of-concept(working in >> production): >> http://lists.freebsd.org/pipermail/freebsd-net/2013-April/035270.html >> >> Another is to change lagg packet accounting: >> http://lists.freebsd.org/pipermail/svn-src-all/2013-April/067570.html >> Again, this is more like HW boxes do (aggregate all counters >> including errors) (and I can't imagine >> what real error we can get from _lagg_). > > >> 4) If we are router, we can do either slooow ip_input() -> >> ip_forward() -> ip_output() cycle or use >> optimized ip_fastfwd() which falls back to 'slow' path for >> multicast/options/local traffic (e.g. >> works exactly like 'data plane' part). >> (Btw, we can consider net.inet.ip.fastforwarding to be turned on by >> default at least for non-IPSEC >> kernels) > > ACK. > >> Here we have to determine if this is local packet or not, e.g. >> F(dst_ip) returning 1 or 0. Currently >> we are simply using standard rlock + hash of iface addresses. >> (And some consumers like ipfw(4) do the same, but without lock). >> We don't need to do this! We can build sorted array of IPv4 addresses >> or other efficient structure >> on every address change and use it unlocked with delayed garbage >> collection (proof-of-concept attached) > > I'm a bit uneasy with unlocked access. On very weakly ordered > architectures > this could trip over cache coherency issues. A rmlock is essentially > for free > in the read case. Well, I'm talking of 1) allocate _new_ memory (unlocked) 2) commit _new_ copy for given address list (rlock) 3) change pointer. As fa as I understand we can read either old or new value 4) use delayed GC (how much should be wait, until deletion) Anyway, protecting (optimized) list with rmlock can do. > >> (There is another thing to discuss: maybe we can do this once >> somewhere in ip_input and mark mbuf as >> 'local/non-local' ? ) > > The problem is packet filters may change the destination address and thus > can invalidate such a lookup. Yes. So ether filter or ip_input() routing should re-inspect packet, exactly like this is done currently (ipfw fwd for IPv4/IPv6 code) > >> 5, 9) Currently we have L3 ingress/egress PFIL hooks protected by >> rmlocks. This is OK. >> >> However, 6) and 7) are not. >> Firewall can use the same pfil lock as reader protection without >> imposing its own lock. currently >> pfil&ipfw code is ready to do this. > > The problem with the global pfil rmlock is the comparatively long time it > is held in a locked state. Also packet filters may have to acquire > additional > locks when they have to modify state tables. Rmlocks are not made for > that > because they pin the thread to the cpu they're currently on. This is > what > Gleb is complaining about. Yes, additional locks is the problem > > My idea is to hold the pfil rmlock only for the lookup of the first/next > packet filter that will run, not for the entire duration. That would > solve > the problem. However packets filter then have to use their own locks > again, > which could be rmlock too. Well, we haven't changed anything yet :) > >> 8) Radix/rt* api. This is probably the worst place in entire stack. >> It is toooo generic, tooo slow >> and buggy (do you use IPv6? you definitely know what I'm talking about). >> A) It really is too generic and assumption that it can be >> (effectively) used for every family is >> wrong. Two examples: >> we don't need to lookup all 128 bits of IPv6 address. Subnets with >> mask >/64 are not used widely >> (actually the only reason to use them are p2p links due to ND >> potential problems). >> One of common solutions is to lookup 64bits, and build another trie >> (or other structure) in case of >> collision. >> Another example is MPLS where we can simply do direct array lookup >> based on ingress label. > > Yes. While we shouldn't throw it out, it should be run as RIB and > allow a much more protocol specific FIB for the hot packet path. > >> B) It is terribly slow (AFAIR luigi@ did some performance management, >> numbers available in one of >> netmap pdfs) > > Again not thaaaat slow but inefficient enough. I've found the paper I was talking about: http://info.iet.unipi.it/~luigi/papers/20120601-dxr.pdf It claims that our radix is able to do 6MPPS/core and it does not scale with number of cores. > >> C) It is not multipath-capable. Stateful (and non-working) multipath >> is definitely not the right way. > > Indeed. > >> 8*) rtentry >> We are doing it wrong. >> Currently _every_ lookup locks/unlocks given rte twice. >> First lock is related to and old-old story for trusting IP redirects >> (and auto-adding host routes >> for them). Hopefully currently it is disabled automatically when you >> turn forwarding on. > > They're disabled. > >> The second one is much more complicated: we are assuming that rte's >> with non-zero refcount value can >> stop egress interface from being destroyed. >> This is wrong (but widely used) assumption. > > Not really. The reason for the refcount is not the ifp reference but > other code parts that may hold direct pointers to the rtentry and do > direct dereferencing to access information in it. Yes, but what information? > >> We can use delayed GC instead of locking for rte's and this won't >> break things more than they are >> broken now (patch attached). > > Nope. Delayed GC is not the way to go here. To do away with rtentry > locking and refcounting we have change rtalloc(9) to return the > information > the caller wants (e.g. ifp, ia, others) and not the rtentry address > anymore. > So instead of rtalloc() we have rtlookup(). It depends on what we want to do next.. My idea (briefly) is to have 1) adjacency/nhops structures describing next hops with rewrite info and list of iface indices to do L2 multipath 2) "rtentry" to have link to array of nhops to do L3 multipath (more or less the same as Cisco CEF and others). And, anyway, we still have to protect from interface departure. > >> We can't do the same for ifp structures since >> a) virtual ones can assume some state in underlying physical NIC >> b) physical ones just _can_ be destroyed (maybe regardless of user >> wants this or not, like: SFP >> being unplugged from NIC) or simply lead to kernel crash due to SW/HW >> inconsistency > > Here I actually believe we can do a GC or stable storage based approach. > Ifp pointers are kept in too many places and properly refcounting it is > very (too) hard. So whenever an interface gets destroyed or disappears > it's callable function pointers are replaced with dummies returning an > error. The ifp in memory will stay for some time and even may be reused Yes. But we are not holding any (relevant) lock while doing actual transmit (e.g. calling if_output after performing L2 rewrite in ether_output) so some cores will see old pointers.. > for another new interface later again (Cisco does it that way in their > IOS). > >> One of possible solution is to implement stable refcounts based on >> PCPU counters, and apply thos >> counters to ifp, but seem to be non-trivial. >> >> >> Another rtalloc(9) problem is the fact that radix is used as both >> 'control plane' and 'data plane' >> structure/api. Some users always want to put more information in rte, >> while others >> want to make rte more compact. We just need _different_ structures >> for that. > > ACK. > >> Feature-rich, lot-of-data control plane one (to store everything we >> want to store, including, for >> example, PID of process originating the route) - current radix can be >> modified to do this. >> And address-family-depended another structure (array, trie, or >> anything) which contains _only_ data >> necessary to put packet on the wire. > > ACK. > >> 11) arpresolve. Currently (this was decoupled in 8.x) we have >> a) ifaddr rlock >> b) lle rlock. >> >> We don't need those locks. >> We need to >> a) make lle layer per-interface instead of global (and this can also >> solve multiple fibs and L2 >> mappings done in fib.0 issue) > > Yes! > >> b) use rtalloc(9)-provided lock instead of separate locking > > No. Interface rmlock. Discussable :) > >> c) actually, we need to do rewrite this layer because >> d) lle actually is the place to do real multipath: > > No, you can do multipath through more than one interface. If lle is > per interface that wont work and is not the right place. > >> briefly, >> you have rte pointing to some special nexthop structure pointing to >> lle, which has the following data: >> num_of_egress_ifaces: [ifindex1, ifindex2, ifindex3] | L2 data to >> prepend to header >> Separate post will follow. > > This should be part of the RIB/FIB and select on of the ifp+nexthops > to return on lookup. Yes. > >> With the following, we can achieve lagg traffic distribution without >> actually using lagg_transmit >> and similar stuff (at least in most common scenarious) > > This seems to be a rather nasty layering violation. Not really. lagg is pure virtual stuff. > >> (for example, TCP output definitely can benefit from this, since we >> can account flowid once for TCP >> session and use in in every mbuf) > > >> So. Imagine we have done all this. How we can estimate the difference? >> >> There was a thread, started a year ago, describing 'stock' >> performance and difference for various >> modifications. >> It is done on 8.x, however I've got similar results on recent 9.x >> >> http://lists.freebsd.org/pipermail/freebsd-net/2012-July/032680.html >> >> Briefly: >> >> 2xE5645 @ Intel 82599 NIC. >> Kernel: FreeBSD-8-S r237994, stock drivers, stock routing, no >> FLOWTABLE, no firewallIxia XM2 >> (traffic generator) <> ix0 (FreeBSD). Ixia sends 64byte IP packets >> from vlan10 (10.100.0.64 - >> 10.100.0.156) to destinations in vlan11 (10.100.1.128 - >> 10.100.1.192). Static arps are configured >> for all destination addresses. Traffic level is slightly above or >> slightly below system performance. >> >> we start from 1.4MPPS (if we are using several routes to minimize >> mutex contention). >> >> My 'current' result for the same test, on same HW, with the following >> modifications: >> >> * 1) ixgbe per-packet ring unlock removed >> * P1) ixgbe is modified to do direct vlan input (so 2,3 are not used) >> * 4) separate lockless in_localip() version >> * 6) - using existing pfil lock >> * 7) using lockless version >> * 8) radix converted to use rmlock instead of rlock. Delayed GC is >> used instead of mutexes >> * 10) - using existing pfil lock >> * 11) using radix lock to do arpresolve(). Not using lle rlock >> >> (so the rmlocks are the only locks used on data path). >> >> Additionally, ipstat counters are converted to PCPU (no real >> performance implications). >> ixgbe does not do per-packet accounting (as in head). >> if_vlan counters are converted to PCPU >> lagg is converted to rmlock, per-packet accounting is removed (using >> stat from underlying interfaces) >> lle hash size is bumped to 1024 instead of 32 (not applicable here, >> but slows things down for large >> L2 domains) >> >> The result is 5.6 MPPS for single port (11 cores) and 6.5MPPS for >> lagg (16 cores), nearly the same >> for HT on and 22 cores. > > That's quite good, but we want more. ;) > >> .. >> while Intel DPDK claims 80MPPS (and 6windgate talks about 160 or so) >> on the same-class hardware and >> _userland_ forwarding. > > Those numbers sound a bit far out. Maybe if the packet isn't touched > or looked at at all in a pure netmap interface to interface bridging > scenario. I don't believe these numbers. http://www.intel.com/content/dam/www/public/us/en/documents/presentation/dpdk-packet-processing-ia-overview-presentation.pdf Luigi talks about very fast L4 lookups in his (and other colleagues) work. Anyway, even simple 8-8-8-8 multi-bit trie can be very fast > >> One of key features making all such products possible (DPDK, netmap, >> packetshader, Cisco SW >> forwarding) - is use of batching instead of process-to-completion model. >> Batching mitigates locking cost, batching does not wash out CPU >> cache, and so on. > > The work has to be done eventually. Batching doesn't relieve from it. > IMHO batch moving is only the last step would should look at. It makes > the stack rather complicated and introduces other issues like packet > latency. > >> So maybe we can consider passing batches from NIC to at least L2 >> layer with netisr? or even up to >> ip_input() ? > > And then? You probably won't win much in the end (if the lock path > is optimized). At least I can firewall them "all at once". Next steps depends on how we can solve egress ifp problem. But yes, this is definitely not the first thing to do. > >> Another question is about making some sort of reliable GC like >> ("passive serialization" or other >> similar not-to-pronounce-words about Linux and lockless objects). > > Rmlocks are our secret weapon and just as good. > >> P.S. Attached patches are 1) for 8.x 2) mostly 'hacks' showing >> roughly how can this be done and what >> benefit can be achieved. > From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 20:02:22 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D762FFB8; Sun, 22 Sep 2013 20:02:22 +0000 (UTC) (envelope-from melifaro@yandex-team.ru) Received: from forward-corp1g.mail.yandex.net (forward-corp1g.mail.yandex.net [IPv6:2a02:6b8:0:1402::10]) by mx1.freebsd.org (Postfix) with ESMTP id 827D92358; Sun, 22 Sep 2013 20:02:22 +0000 (UTC) Received: from smtpcorp4.mail.yandex.net (smtpcorp4.mail.yandex.net [95.108.252.2]) by forward-corp1g.mail.yandex.net (Yandex) with ESMTP id C54B9366009A; Mon, 23 Sep 2013 00:02:20 +0400 (MSK) Received: from smtpcorp4.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp4.mail.yandex.net (Yandex) with ESMTP id 2EF512C0337; Mon, 23 Sep 2013 00:02:20 +0400 (MSK) Received: from dhcp170-36-red.yandex.net (dhcp170-36-red.yandex.net [95.108.170.36]) by smtpcorp4.mail.yandex.net (nwsmtp/Yandex) with ESMTP id sZ8d66c12V-2KiurqgF; Mon, 23 Sep 2013 00:02:20 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1379880140; bh=ckqWsTOIe/Kq2fUVwOM8tdSMGh1wrt74DVOIcVSoROQ=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=UqhwclJ5XqEh3ojfScL1sDTZYC6IyJM1t6cWll5wv/fCnjrOTu4KyAfG/wQzY9rkf FNwVc0pgaxTowGHI03/Hgr3+l9ByEYnLs4ZAX36XWRwghuhfnKgXNNKA8RXVJZ8xyb lztFaDePItV+hzB91UQp9tJrQKpHpBB8VWUc+YRI= Authentication-Results: smtpcorp4.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <523F4C8D.6080903@yandex-team.ru> Date: Mon, 23 Sep 2013 00:01:17 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130824 Thunderbird/17.0.8 MIME-Version: 1.0 To: Slawa Olhovchenkov Subject: Re: Network stack changes References: <521E41CB.30700@yandex-team.ru> <521E78B0.6080709@freebsd.org> <20130829013241.GB70584@zxy.spb.ru> In-Reply-To: <20130829013241.GB70584@zxy.spb.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 22 Sep 2013 22:24:56 +0000 Cc: adrian@freebsd.org, Andre Oppermann , freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org, luigi@freebsd.org, ae@FreeBSD.org, Gleb Smirnoff , FreeBSD Net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 20:02:22 -0000 On 29.08.2013 05:32, Slawa Olhovchenkov wrote: > On Thu, Aug 29, 2013 at 12:24:48AM +0200, Andre Oppermann wrote: > >>> .. >>> while Intel DPDK claims 80MPPS (and 6windgate talks about 160 or so) on the same-class hardware and >>> _userland_ forwarding. >> Those numbers sound a bit far out. Maybe if the packet isn't touched >> or looked at at all in a pure netmap interface to interface bridging >> scenario. I don't believe these numbers. > 80*64*8 = 40.960 Gb/s > May be DCA? And use CPU with 40 PCIe lane and 4 memory chanell. Intel introduces DDIO instead of DCA: http://www.intel.com/content/www/us/en/io/direct-data-i-o.html (and it seems DCA does not help much): https://www.myricom.com/software/myri10ge/790-how-do-i-enable-intel-direct-cache-access-dca-with-the-linux-myri10ge-driver.html https://www.myricom.com/software/myri10ge/783-how-do-i-get-the-best-performance-with-my-myri-10g-network-adapters-on-a-host-that-supports-intel-data-direct-i-o-ddio.html (However, DPDK paper notes DDIO is of signifficant helpers) From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 20:13:11 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 956362E0; Sun, 22 Sep 2013 20:13:11 +0000 (UTC) (envelope-from melifaro@yandex-team.ru) Received: from forward-corp1f.mail.yandex.net (forward-corp1f.mail.yandex.net [IPv6:2a02:6b8:0:801::10]) by mx1.freebsd.org (Postfix) with ESMTP id 0AEAD23F6; Sun, 22 Sep 2013 20:13:11 +0000 (UTC) Received: from smtpcorp4.mail.yandex.net (smtpcorp4.mail.yandex.net [95.108.252.2]) by forward-corp1f.mail.yandex.net (Yandex) with ESMTP id 9743C2420022; Mon, 23 Sep 2013 00:13:07 +0400 (MSK) Received: from smtpcorp4.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp4.mail.yandex.net (Yandex) with ESMTP id CB2EB2C032B; Mon, 23 Sep 2013 00:13:06 +0400 (MSK) Received: from dhcp170-36-red.yandex.net (dhcp170-36-red.yandex.net [95.108.170.36]) by smtpcorp4.mail.yandex.net (nwsmtp/Yandex) with ESMTP id zPyEtHgBff-D6iqY3ni; Mon, 23 Sep 2013 00:13:06 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1379880786; bh=lc1BwebjnC7LP0vJSar/INsu9fAdxzJlXjhZRROe3wc=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=TiFhbX28OlyJCId0ZK+VwWQp36YyQ+h5WJh3zgC8KN+0g0HD9q83tsYSnh8yv3F8L xX3AllLMhINn8mDOk4NZSsUIRGpnNhckim3gXJVni//gJQRXbaQZLY79Q45TEGh/pn 8RzHV1D9iadYLhu84SNJrkjRmQNZV1G2ug+v8DQo= Authentication-Results: smtpcorp4.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <523F4F14.9090404@yandex-team.ru> Date: Mon, 23 Sep 2013 00:12:04 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130824 Thunderbird/17.0.8 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: Network stack changes References: <521E41CB.30700@yandex-team.ru> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 22 Sep 2013 22:25:15 +0000 Cc: Luigi Rizzo , Andre Oppermann , "freebsd-hackers@freebsd.org" , FreeBSD Net , "Andrey V. Elsukov" , Gleb Smirnoff , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 20:13:11 -0000 On 29.08.2013 15:49, Adrian Chadd wrote: > Hi, Hello Adrian! I'm very sorry for the looong reply. > > There's a lot of good stuff to review here, thanks! > > Yes, the ixgbe RX lock needs to die in a fire. It's kinda pointless to > keep locking things like that on a per-packet basis. We should be able > to do this in a cleaner way - we can defer RX into a CPU pinned > taskqueue and convert the interrupt handler to a fast handler that > just schedules that taskqueue. We can ignore the ithread entirely here. > > What do you think? Well, it sounds good :) But performance numbers and Jack opinion is more important :) Are you going to Malta? > > Totally pie in the sky handwaving at this point: > > * create an array of mbuf pointers for completed mbufs; > * populate the mbuf array; > * pass the array up to ether_demux(). > > For vlan handling, it may end up populating its own list of mbufs to > push up to ether_demux(). So maybe we should extend the API to have a > bitmap of packets to actually handle from the array, so we can pass up > a larger array of mbufs, note which ones are for the destination and > then the upcall can mark which frames its consumed. > > I specifically wonder how much work/benefit we may see by doing: > > * batching packets into lists so various steps can batch process > things rather than run to completion; > * batching the processing of a list of frames under a single lock > instance - eg, if the forwarding code could do the forwarding lookup > for 'n' packets under a single lock, then pass that list of frames up > to inet_pfil_hook() to do the work under one lock, etc, etc. I'm thinking the same way, but we're stuck with 'forwarding lookup' due to problem with egress interface pointer, as I mention earlier. However it is interesting to see how much it helps, regardless of locking. Currently I'm thinking that we should try to change radix to something different (it seems that it can be checked fast) and see what happened. Luigi's performance numbers for our radix are too awful, and there is a patch implementing alternative trie: http://info.iet.unipi.it/~luigi/papers/20120601-dxr.pdf http://www.nxlab.fer.hr/dxr/stable_8_20120824.diff > > Here, the processing would look less like "grab lock and process to > completion" and more like "mark and sweep" - ie, we have a list of > frames that we mark as needing processing and mark as having been > processed at each layer, so we know where to next dispatch them. > > I still have some tool coding to do with PMC before I even think about > tinkering with this as I'd like to measure stuff like per-packet > latency as well as top-level processing overhead (ie, > CPU_CLK_UNHALTED.THREAD_P / lagg0 TX bytes/pkts, RX bytes/pkts, NIC > interrupts on that core, etc.) That will be great to see! > > Thanks, > > > > -adrian > From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 20:16:21 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 53BB2667; Sun, 22 Sep 2013 20:16:21 +0000 (UTC) (envelope-from melifaro@yandex-team.ru) Received: from forward-corp1e.mail.yandex.net (forward-corp1e.mail.yandex.net [IPv6:2a02:6b8:0:202::10]) by mx1.freebsd.org (Postfix) with ESMTP id EC30D242B; Sun, 22 Sep 2013 20:16:20 +0000 (UTC) Received: from smtpcorp4.mail.yandex.net (smtpcorp4.mail.yandex.net [95.108.252.2]) by forward-corp1e.mail.yandex.net (Yandex) with ESMTP id 95BAF640D0A; Mon, 23 Sep 2013 00:16:17 +0400 (MSK) Received: from smtpcorp4.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp4.mail.yandex.net (Yandex) with ESMTP id 581372C032B; Mon, 23 Sep 2013 00:16:16 +0400 (MSK) Received: from dhcp170-36-red.yandex.net (dhcp170-36-red.yandex.net [95.108.170.36]) by smtpcorp4.mail.yandex.net (nwsmtp/Yandex) with ESMTP id jtjruyBg15-GGiaTnd3; Mon, 23 Sep 2013 00:16:16 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1379880976; bh=gvzJN62AvNEgGoGNHIYfEvCzLEcKQ9CJd4v9pZemI00=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=Gc+87YUS6ZIHr6f8OibtRuhzdovntf4M5P1jnv32kAeNRLqwz3/oJ2UxeURkPMrID rfYovH/KAOgmb5HMFh3SgBd2fcOQME6BrBubz6ELgGGcqEiBEEevu5QbGoqcve1PRb tuPOPlllk/24RD9I7z4/G5jCBW9yrIuBlc0AVoao= Authentication-Results: smtpcorp4.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <523F4FD1.6060807@yandex-team.ru> Date: Mon, 23 Sep 2013 00:15:13 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130824 Thunderbird/17.0.8 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Olivier_Cochard-Labb=E9?= Subject: Re: Network stack changes References: <521E41CB.30700@yandex-team.ru> <6BDA4619-783C-433E-9819-A7EAA0BD3299@neville-neil.com> <20130914142802.GC71010@onelab2.iet.unipi.it> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Sun, 22 Sep 2013 22:25:31 +0000 Cc: Adrian Chadd , Andre Oppermann , "freebsd-hackers@freebsd.org" , George Neville-Neil , "freebsd-arch@freebsd.org" , Luigi Rizzo , "Andrey V. Elsukov" , Gleb Smirnoff , FreeBSD Net , Luigi Rizzo X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 20:16:21 -0000 On 14.09.2013 22:49, Olivier Cochard-Labbé wrote: > On Sat, Sep 14, 2013 at 4:28 PM, Luigi Rizzo wrote: >> IXIA ? For the timescales we need to address we don't need an IXIA, >> a netmap sender is more than enough >> > The great netmap generates only one IP flow (same src/dst IP and same > src/dst port). > This don't permit to test multi-queue NIC (or SMP packet-filter) on a > simple lab like this: > netmap sender => freebsd router => netmap receiver I've got the variant which is capable on doing linerate pcap replays on single queue. (However this is true for small pcaps only) > > Regards, > > Olivier From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 22:06:04 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id ACEF0C12; Sun, 22 Sep 2013 22:06:04 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 68C9E29BF; Sun, 22 Sep 2013 22:06:04 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1VNrp0-000Llg-HY; Mon, 23 Sep 2013 02:08:06 +0400 Date: Mon, 23 Sep 2013 02:08:06 +0400 From: Slawa Olhovchenkov To: "Alexander V. Chernikov" Subject: Re: Network stack changes Message-ID: <20130922220806.GK3796@zxy.spb.ru> References: <521E41CB.30700@yandex-team.ru> <521E78B0.6080709@freebsd.org> <20130829013241.GB70584@zxy.spb.ru> <523F4C8D.6080903@yandex-team.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <523F4C8D.6080903@yandex-team.ru> User-Agent: Mutt/1.5.21 (2010-09-15) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-Mailman-Approved-At: Sun, 22 Sep 2013 22:25:40 +0000 Cc: adrian@freebsd.org, Andre Oppermann , freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org, luigi@freebsd.org, ae@FreeBSD.org, Gleb Smirnoff , FreeBSD Net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 22:06:04 -0000 On Mon, Sep 23, 2013 at 12:01:17AM +0400, Alexander V. Chernikov wrote: > On 29.08.2013 05:32, Slawa Olhovchenkov wrote: > > On Thu, Aug 29, 2013 at 12:24:48AM +0200, Andre Oppermann wrote: > > > >>> .. > >>> while Intel DPDK claims 80MPPS (and 6windgate talks about 160 or so) on the same-class hardware and > >>> _userland_ forwarding. > >> Those numbers sound a bit far out. Maybe if the packet isn't touched > >> or looked at at all in a pure netmap interface to interface bridging > >> scenario. I don't believe these numbers. > > 80*64*8 = 40.960 Gb/s > > May be DCA? And use CPU with 40 PCIe lane and 4 memory chanell. > Intel introduces DDIO instead of DCA: > http://www.intel.com/content/www/us/en/io/direct-data-i-o.html > (and it seems DCA does not help much): > https://www.myricom.com/software/myri10ge/790-how-do-i-enable-intel-direct-cache-access-dca-with-the-linux-myri10ge-driver.html > https://www.myricom.com/software/myri10ge/783-how-do-i-get-the-best-performance-with-my-myri-10g-network-adapters-on-a-host-that-supports-intel-data-direct-i-o-ddio.html > > (However, DPDK paper notes DDIO is of signifficant helpers) Ha, Intel paper say SMT is signifficant better HT. In real word -- same shit. For network application, if buffring need more then L3 cache, what happening? May be some bad things... From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 23:18:30 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D3E179CB; Sun, 22 Sep 2013 23:18:30 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AAF2B2DAE; Sun, 22 Sep 2013 23:18:30 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1VNsv7-00017t-6T; Sun, 22 Sep 2013 23:18:29 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r8MNIQom022737; Sun, 22 Sep 2013 17:18:26 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX19V8/gGvFRosNvZbXNPKZve Subject: The right way to invoke sh from a freebsd makefile? From: Ian Lepore To: FreeBSD Hackers , freebsd-current Content-Type: text/plain; charset="us-ascii" Date: Sun, 22 Sep 2013 17:18:25 -0600 Message-ID: <1379891905.1197.115.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 23:18:30 -0000 What's the right way to launch the bourne shell from a makefile? I had assumed the ${SHELL} variable would be set to "the right" copy of /bin/sh (like maybe the one in tmp or legacy at various stages). It appears that that's not the case, and ${SHELL} is whatever comes from the environment, which can lead to using csh or bash or whatever. I see some of our makefiles use just a bare "sh" which seems reasonable to me, but I don't want to glitch this in src/include/Makefile again. The goal is to run a script in src/include/Makefile by launching sh with the script name (as opposed to launching the script and letting the #! do its thing, which doesn't work if the source dir is mounted noexec). -- Ian From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 23:25:51 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0DAA0EE7 for ; Sun, 22 Sep 2013 23:25:51 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E0F542E82 for ; Sun, 22 Sep 2013 23:25:50 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r8MNPobU099757 for ; Sun, 22 Sep 2013 23:25:50 GMT (envelope-from bdrewery@freefall.freebsd.org) Received: (from bdrewery@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r8MNPo26099753 for freebsd-hackers@FreeBSD.org; Sun, 22 Sep 2013 23:25:50 GMT (envelope-from bdrewery) Received: (qmail 71530 invoked from network); 22 Sep 2013 18:25:46 -0500 Received: from unknown (HELO ?10.10.0.24?) (freebsd@shatow.net@10.10.0.24) by sweb.xzibition.com with ESMTPA; 22 Sep 2013 18:25:46 -0500 Message-ID: <523F7C78.8080603@FreeBSD.org> Date: Sun, 22 Sep 2013 18:25:44 -0500 From: Bryan Drewery Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Ian Lepore Subject: Re: The right way to invoke sh from a freebsd makefile? References: <1379891905.1197.115.camel@revolution.hippie.lan> In-Reply-To: <1379891905.1197.115.camel@revolution.hippie.lan> X-Enigmail-Version: 1.5.2 OpenPGP: id=3C9B0CF9; url=http://www.shatow.net/bryan/bryan.asc Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="X1viU4XMpO19OV2vj6IXJPqJhrVikWAOM" Cc: FreeBSD Hackers , freebsd-current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 23:25:51 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --X1viU4XMpO19OV2vj6IXJPqJhrVikWAOM Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 9/22/2013 6:18 PM, Ian Lepore wrote: > What's the right way to launch the bourne shell from a makefile? I had= > assumed the ${SHELL} variable would be set to "the right" copy > of /bin/sh (like maybe the one in tmp or legacy at various stages). It= > appears that that's not the case, and ${SHELL} is whatever comes from > the environment, which can lead to using csh or bash or whatever. >=20 > I see some of our makefiles use just a bare "sh" which seems reasonable= > to me, but I don't want to glitch this in src/include/Makefile again. > The goal is to run a script in src/include/Makefile by launching sh wit= h > the script name (as opposed to launching the script and letting the #! > do its thing, which doesn't work if the source dir is mounted noexec). >=20 > -- Ian >=20 Grepping the Makefiles in the tree, 'sh' is the very common. I see around 157 users of this pattern. 13 use /bin/sh directly. Also consider that it is highly likely, if not required, that a /bin/sh will exist. Calling 'sh' specifically is definitely more proper than ${SHELL} since it is an sh script. --=20 Regards, Bryan Drewery --X1viU4XMpO19OV2vj6IXJPqJhrVikWAOM Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSP3x4AAoJEG54KsA8mwz5kSAP/2wm6+efiXjPZAKG2AYntqPX seA40sfTGd0p81eW32HroEv+r3bg0FXL4CnRoIduQQ68tPLvlVA8/V/4JnD/LE3L EiIIru5sRvW1QodHTRVB572jknocFULFg7VquLeEc6mXCmjuvdslHhgnJk84/MZa ZMrhyQzw+HmGzkVGACqFC98FIQsRASwEWxBihz6C2n0naVUo5DhPfjio/uKOtq5+ QBUMvICAAA0K2S2foPP1fV8tyefQxWl43uGxFQqj9GDmU7xAYCjkC0TmJ7cHgCcI h39jGNABCQ8pR8wwEdI8JvbF9XaqukAPuuU3yov0MfWi0waKPWZaX6rWdtLeMpkP JXfBovXR3gyWE5LjYTqja4NFmhbA4bzq/GJhXa5IB4K0XcVOQhj5Eb/MdfniXADz bsspdkR+NDNtNmHB9tAdjE/Bvh0KoRc4+BhF2aQCua5yHrVQPaAJP3g0AvquD1Ux j8u9pXbrTx6dLHdhmASmD/C/lB6L0HMjpGC1peUweZGl7a5uR/Ln+QegokdJuPbk BnGSZb0eKxp28auIMbAPlGPziEAouXprLbJ4mihtP5WT9zwUeZmrmUzNXluwEofA 59q3wneeYnwFOgRnTYO0qKqAqKW/GLaMjl/AIuwmMn7LemCVKmVRQE6A/mw4x/OK ICSdGlW0nZC+1oLmllDG =bH97 -----END PGP SIGNATURE----- --X1viU4XMpO19OV2vj6IXJPqJhrVikWAOM-- From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 23:27:13 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1CFE21D2; Sun, 22 Sep 2013 23:27:13 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [208.86.227.67]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E59162ED5; Sun, 22 Sep 2013 23:27:12 +0000 (UTC) Received: from glenbarber.us (c-71-224-221-174.hsd1.nj.comcast.net [71.224.221.174]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id 9E08B4ED2; Sun, 22 Sep 2013 23:27:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us 9E08B4ED2 Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Sun, 22 Sep 2013 19:27:10 -0400 From: Glen Barber To: Ian Lepore Subject: Re: The right way to invoke sh from a freebsd makefile? Message-ID: <20130922232710.GD2336@glenbarber.us> References: <1379891905.1197.115.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="Km1U/tdNT/EmXiR1" Content-Disposition: inline In-Reply-To: <1379891905.1197.115.camel@revolution.hippie.lan> X-Operating-System: FreeBSD 10.0-ALPHA2 amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Hackers , freebsd-current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 23:27:13 -0000 --Km1U/tdNT/EmXiR1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 22, 2013 at 05:18:25PM -0600, Ian Lepore wrote: > What's the right way to launch the bourne shell from a makefile? I had > assumed the ${SHELL} variable would be set to "the right" copy > of /bin/sh (like maybe the one in tmp or legacy at various stages). It > appears that that's not the case, and ${SHELL} is whatever comes from > the environment, which can lead to using csh or bash or whatever. >=20 > I see some of our makefiles use just a bare "sh" which seems reasonable > to me, but I don't want to glitch this in src/include/Makefile again. > The goal is to run a script in src/include/Makefile by launching sh with > the script name (as opposed to launching the script and letting the #! > do its thing, which doesn't work if the source dir is mounted noexec). >=20 I think BUILDENV_SHELL is what you are looking for. For this specific case, I think instead of '#!/bin/sh', maybe '#!/usr/bin/env sh' may be preferable. Glen --Km1U/tdNT/EmXiR1 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQEcBAEBCAAGBQJSP3zOAAoJEFJPDDeguUajdYEH/AqxeG19/GGa9zYHLmaqxr+4 sjDJ9Y++f99y+1XLTRdjwPZWRY/8g/nMpXEs42y+HP1Ap2CTdlyCom+i//cQUQSL DKGfJosGErZpuEIxXHXeou7LKg1mNtKNHe+JW0O5vC9GWOFB+9D1y6iR9RHX9V/l LLQIC9h7jRJAoqLMJ24f2y6zgYWjaMqEvSHNS6+756y126QxjY058bTHDg1stOIE WoLRaYydLhDWp80go6m0Os34VRks/HgcKOCCi6RBQi7BV/pu7303vchamSOxCW5W ACOjGtXBP7jM44MQzHf6KmKLeoBffTSqY5VzZyHxpTYqTmwPpTNi7/YdMtOhUT8= =PS+8 -----END PGP SIGNATURE----- --Km1U/tdNT/EmXiR1-- From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 23:38:02 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4DD54500; Sun, 22 Sep 2013 23:38:02 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 216122F4E; Sun, 22 Sep 2013 23:38:01 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1VNtDu-000O9n-VF; Sun, 22 Sep 2013 23:37:55 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r8MNbpRN022766; Sun, 22 Sep 2013 17:37:51 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/NtI61I85ts8+D2uTeLGV/ Subject: Re: The right way to invoke sh from a freebsd makefile? From: Ian Lepore To: Glen Barber In-Reply-To: <20130922232710.GD2336@glenbarber.us> References: <1379891905.1197.115.camel@revolution.hippie.lan> <20130922232710.GD2336@glenbarber.us> Content-Type: text/plain; charset="us-ascii" Date: Sun, 22 Sep 2013 17:37:51 -0600 Message-ID: <1379893071.1197.119.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers , freebsd-current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 23:38:02 -0000 On Sun, 2013-09-22 at 19:27 -0400, Glen Barber wrote: > On Sun, Sep 22, 2013 at 05:18:25PM -0600, Ian Lepore wrote: > > What's the right way to launch the bourne shell from a makefile? I had > > assumed the ${SHELL} variable would be set to "the right" copy > > of /bin/sh (like maybe the one in tmp or legacy at various stages). It > > appears that that's not the case, and ${SHELL} is whatever comes from > > the environment, which can lead to using csh or bash or whatever. > > > > I see some of our makefiles use just a bare "sh" which seems reasonable > > to me, but I don't want to glitch this in src/include/Makefile again. > > The goal is to run a script in src/include/Makefile by launching sh with > > the script name (as opposed to launching the script and letting the #! > > do its thing, which doesn't work if the source dir is mounted noexec). > > > > I think BUILDENV_SHELL is what you are looking for. For this specific > case, I think instead of '#!/bin/sh', maybe '#!/usr/bin/env sh' may be > preferable. > > Glen > No, BUILDENV_SHELL is a special thing... it's used when you "make buildenv" to chroot into a cross-build environment to work interactively. I added that long ago because I can't live in a csh shell (I mean, I can't do anything, I'm totally lost), and I wanted a way to have "make buildenv" put me right into bash (of course, you have to have bash in the chroot). The flavor of hashbang to use shouldn't matter, since what I'm after here is launching the shell to run the script without using the hashbang mechanism. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 23:45:59 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5F9A485B; Sun, 22 Sep 2013 23:45:59 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [IPv6:2607:fc50:1:2300:1001:1001:1001:face]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3229F2FF2; Sun, 22 Sep 2013 23:45:59 +0000 (UTC) Received: from glenbarber.us (c-71-224-221-174.hsd1.nj.comcast.net [71.224.221.174]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id 1F1A4407A; Sun, 22 Sep 2013 23:45:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us 1F1A4407A Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Sun, 22 Sep 2013 19:45:55 -0400 From: Glen Barber To: Ian Lepore Subject: Re: The right way to invoke sh from a freebsd makefile? Message-ID: <20130922234555.GE2336@glenbarber.us> References: <1379891905.1197.115.camel@revolution.hippie.lan> <20130922232710.GD2336@glenbarber.us> <1379893071.1197.119.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="5xSkJheCpeK0RUEJ" Content-Disposition: inline In-Reply-To: <1379893071.1197.119.camel@revolution.hippie.lan> X-Operating-System: FreeBSD 10.0-ALPHA2 amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Hackers , freebsd-current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 23:45:59 -0000 --5xSkJheCpeK0RUEJ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 22, 2013 at 05:37:51PM -0600, Ian Lepore wrote: > On Sun, 2013-09-22 at 19:27 -0400, Glen Barber wrote: > > On Sun, Sep 22, 2013 at 05:18:25PM -0600, Ian Lepore wrote: > > > What's the right way to launch the bourne shell from a makefile? I h= ad > > > assumed the ${SHELL} variable would be set to "the right" copy > > > of /bin/sh (like maybe the one in tmp or legacy at various stages). = It > > > appears that that's not the case, and ${SHELL} is whatever comes from > > > the environment, which can lead to using csh or bash or whatever. > > >=20 > > > I see some of our makefiles use just a bare "sh" which seems reasonab= le > > > to me, but I don't want to glitch this in src/include/Makefile again. > > > The goal is to run a script in src/include/Makefile by launching sh w= ith > > > the script name (as opposed to launching the script and letting the #! > > > do its thing, which doesn't work if the source dir is mounted noexec). > > >=20 > >=20 > > I think BUILDENV_SHELL is what you are looking for. For this specific > > case, I think instead of '#!/bin/sh', maybe '#!/usr/bin/env sh' may be > > preferable. > >=20 > > Glen > >=20 >=20 > No, BUILDENV_SHELL is a special thing... it's used when you "make > buildenv" to chroot into a cross-build environment to work > interactively. I added that long ago because I can't live in a csh > shell (I mean, I can't do anything, I'm totally lost), and I wanted a > way to have "make buildenv" put me right into bash (of course, you have > to have bash in the chroot). >=20 Ah, right. Thanks for the sanity check. > The flavor of hashbang to use shouldn't matter, since what I'm after > here is launching the shell to run the script without using the hashbang > mechanism. >=20 You can hard-code /bin/sh directly, but what I was getting at with the '#!/usr/bin/env sh' is that the 'sh' interpreter of the build environment could be used (instead of /bin/sh directly). Then you don't need to worry about the path to sh(1). Glen --5xSkJheCpeK0RUEJ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQEcBAEBCAAGBQJSP4EzAAoJEFJPDDeguUajQWUIAKaWdeFQxJahvN2CEgAFGhB/ Ftoj1nas+yeDG3jRCEgAHF9jdiyAPZKF/UGFFtJTdH7NfQRpaO7fKjucKoTvAa5k atypjFpIYVRmyJGgI5KkwvMMYtjhUoYopkyZCi2SLZlFXhVTFInKZxsHu6lJwXkW yGAo9KRKeFGFnu3XcF02OWlpaE5yHocvzrgCM7D1mrLyRzwxxpLjyRqb4MrZ08uJ /JkMODxkpiRASPQERoYfL3M2B5RLoZsYvPG1vLI/ehl+CnhUTIQehXDQGrQCMkAA zkyrQUWggv2vyiV/6EQv3lIzrKgrJITMJnjUmrcVdvLffSU/gAn3huLKeZkwMbY= =rtRm -----END PGP SIGNATURE----- --5xSkJheCpeK0RUEJ-- From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 22 23:56:12 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DA0F6D44; Sun, 22 Sep 2013 23:56:11 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AD88D2092; Sun, 22 Sep 2013 23:56:11 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1VNtVa-0007xk-H9; Sun, 22 Sep 2013 23:56:10 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r8MNu7PR022792; Sun, 22 Sep 2013 17:56:07 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/eINHNuKI1Do/W+Dt8pzPd Subject: Re: The right way to invoke sh from a freebsd makefile? From: Ian Lepore To: Glen Barber In-Reply-To: <20130922234555.GE2336@glenbarber.us> References: <1379891905.1197.115.camel@revolution.hippie.lan> <20130922232710.GD2336@glenbarber.us> <1379893071.1197.119.camel@revolution.hippie.lan> <20130922234555.GE2336@glenbarber.us> Content-Type: text/plain; charset="us-ascii" Date: Sun, 22 Sep 2013 17:56:07 -0600 Message-ID: <1379894167.1197.126.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers , freebsd-current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 23:56:12 -0000 On Sun, 2013-09-22 at 19:45 -0400, Glen Barber wrote: > On Sun, Sep 22, 2013 at 05:37:51PM -0600, Ian Lepore wrote: > > On Sun, 2013-09-22 at 19:27 -0400, Glen Barber wrote: > > > On Sun, Sep 22, 2013 at 05:18:25PM -0600, Ian Lepore wrote: > > > > What's the right way to launch the bourne shell from a makefile? I had > > > > assumed the ${SHELL} variable would be set to "the right" copy > > > > of /bin/sh (like maybe the one in tmp or legacy at various stages). It > > > > appears that that's not the case, and ${SHELL} is whatever comes from > > > > the environment, which can lead to using csh or bash or whatever. > > > > > > > > I see some of our makefiles use just a bare "sh" which seems reasonable > > > > to me, but I don't want to glitch this in src/include/Makefile again. > > > > The goal is to run a script in src/include/Makefile by launching sh with > > > > the script name (as opposed to launching the script and letting the #! > > > > do its thing, which doesn't work if the source dir is mounted noexec). > > > > > > > > > > I think BUILDENV_SHELL is what you are looking for. For this specific > > > case, I think instead of '#!/bin/sh', maybe '#!/usr/bin/env sh' may be > > > preferable. > > > > > > Glen > > > > > > > No, BUILDENV_SHELL is a special thing... it's used when you "make > > buildenv" to chroot into a cross-build environment to work > > interactively. I added that long ago because I can't live in a csh > > shell (I mean, I can't do anything, I'm totally lost), and I wanted a > > way to have "make buildenv" put me right into bash (of course, you have > > to have bash in the chroot). > > > > Ah, right. Thanks for the sanity check. > > > The flavor of hashbang to use shouldn't matter, since what I'm after > > here is launching the shell to run the script without using the hashbang > > mechanism. > > > > You can hard-code /bin/sh directly, but what I was getting at with the > '#!/usr/bin/env sh' is that the 'sh' interpreter of the build > environment could be used (instead of /bin/sh directly). Then you don't > need to worry about the path to sh(1). > > Glen > My point is that the #! isn't used at all in this case, it doesn't matter what's there. Try this... echo "echo foo" >/tmp/foo sh /tmp/foo Not only does it not need the hashbang, the script doesn't even have to be executable when you launch sh and name a script on the command line, which is just what's needed to run a script from a directory mounted with the noexec flag. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 00:02:50 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DF6E010F; Mon, 23 Sep 2013 00:02:50 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [IPv6:2607:fc50:1:2300:1001:1001:1001:face]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B14DB2107; Mon, 23 Sep 2013 00:02:50 +0000 (UTC) Received: from glenbarber.us (c-71-224-221-174.hsd1.nj.comcast.net [71.224.221.174]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id A182941E4; Mon, 23 Sep 2013 00:02:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us A182941E4 Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Sun, 22 Sep 2013 20:02:48 -0400 From: Glen Barber To: Ian Lepore Subject: Re: The right way to invoke sh from a freebsd makefile? Message-ID: <20130923000248.GG2336@glenbarber.us> References: <1379891905.1197.115.camel@revolution.hippie.lan> <20130922232710.GD2336@glenbarber.us> <1379893071.1197.119.camel@revolution.hippie.lan> <20130922234555.GE2336@glenbarber.us> <1379894167.1197.126.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="8S1fMsFYqgBC+BN/" Content-Disposition: inline In-Reply-To: <1379894167.1197.126.camel@revolution.hippie.lan> X-Operating-System: FreeBSD 10.0-ALPHA2 amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Hackers , freebsd-current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 00:02:51 -0000 --8S1fMsFYqgBC+BN/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 22, 2013 at 05:56:07PM -0600, Ian Lepore wrote: > > You can hard-code /bin/sh directly, but what I was getting at with the > > '#!/usr/bin/env sh' is that the 'sh' interpreter of the build > > environment could be used (instead of /bin/sh directly). Then you don't > > need to worry about the path to sh(1). > >=20 >=20 > My point is that the #! isn't used at all in this case, it doesn't > matter what's there. Try this... >=20 > echo "echo foo" >/tmp/foo > sh /tmp/foo >=20 > Not only does it not need the hashbang, the script doesn't even have to > be executable when you launch sh and name a script on the command line, > which is just what's needed to run a script from a directory mounted > with the noexec flag. >=20 Ah - maybe it's just late. I see what you mean now. Thanks. Glen --8S1fMsFYqgBC+BN/ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQEcBAEBCAAGBQJSP4UoAAoJEFJPDDeguUaj91EH/1KMDVCKVlYuTsETjY/nf9v4 VJVPpStKKBrGfs1VtIu9Z+s202iBgRVZwu7qp9kbRUbaTP4h+4cQlgLlMWClCMMM x+M1wHJ4sYAmbec8yQCppXJOiu5pGJy37mZSsfRVaz6695PrOlmIfOXtGACY4ajZ st/EJRaz6gP8HFZe91kp3KW8cCZzxiLRPOdGw/m6DDjf9oAyywO8d9WWvpZs00IA 3M17XGR81s3BBGBnN4gghJEeRjtid8qjkI/32oUqUTkYy8An/lR1HIlpDq5T0ihX 2Qlv5Y/MhwQG0oP7RIwbMXEPyQitnj9GWQ62wS2rCTzcj6fObma1E3uOR0qJ7ts= =/mOq -----END PGP SIGNATURE----- --8S1fMsFYqgBC+BN/-- From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 04:42:35 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D0597AD1; Mon, 23 Sep 2013 04:42:35 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x236.google.com (mail-wg0-x236.google.com [IPv6:2a00:1450:400c:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 836B62B50; Mon, 23 Sep 2013 04:42:34 +0000 (UTC) Received: by mail-wg0-f54.google.com with SMTP id m15so2690852wgh.9 for ; Sun, 22 Sep 2013 21:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=P2w1WbOvUNizXn7ZsZS2TBooYqeg4cGtYAdYuHYhYXM=; b=QOwrZB5l04e+6jyPEqzodo92zbc+lgK+Acasb9IebLx+0pMt++4YSkD1Y69XGeJN47 O5q78g4joGcHZCXb4Sn5q/+b8pgFZzoqCX3HbDHnjWLGeLaeIzvsMPRMJ2QjUd+DPgI4 kQeWEDgL2puChSMNZNc0GP+Yrc5jMlRtlX16RU2tYuX0zwZ74ydipyKtRv7zVzP5SD2i QNIdpI9rcwdFWjQBTbPx24KSQcZX+2ykEAL8KEFOJpfUIDs6zPXLWk25mKbutnQm3DZW SKWgHGVY1h0IpgLyJvTbTq+B4vDL1URAj6tEcZU0Ne9IVvTY0BRjmiX7vQU/6ObMDegy GoVA== MIME-Version: 1.0 X-Received: by 10.180.10.136 with SMTP id i8mr11840167wib.46.1379911352875; Sun, 22 Sep 2013 21:42:32 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.73.133 with HTTP; Sun, 22 Sep 2013 21:42:32 -0700 (PDT) In-Reply-To: <523F4F14.9090404@yandex-team.ru> References: <521E41CB.30700@yandex-team.ru> <523F4F14.9090404@yandex-team.ru> Date: Sun, 22 Sep 2013 21:42:32 -0700 X-Google-Sender-Auth: Xk5o-T54T2C85UHlUNeXvbPizgo Message-ID: Subject: Re: Network stack changes From: Adrian Chadd To: "Alexander V. Chernikov" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Luigi Rizzo , Andre Oppermann , "freebsd-hackers@freebsd.org" , FreeBSD Net , "Andrey V. Elsukov" , Gleb Smirnoff , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 04:42:36 -0000 Hi! On 22 September 2013 13:12, Alexander V. Chernikov wrote: > I'm thinking the same way, but we're stuck with 'forwarding lookup' due > to problem with egress interface pointer, as I mention earlier. However it > is interesting to see how much it helps, regardless of locking. > > Currently I'm thinking that we should try to change radix to something > different (it seems that it can be checked fast) and see what happened. > Luigi's performance numbers for our radix are too awful, and there is a > patch implementing alternative trie: > http://info.iet.unipi.it/~**luigi/papers/20120601-dxr.pdf > http://www.nxlab.fer.hr/dxr/**stable_8_20120824.diff > > So, I can make educated guesses about why this is better for forwarding workloads. I'd like to characterize it though. So, what's it doing that's better? better locking? better caching behaviour? less memory lookups? etc. Thanks, -adrian From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 06:23:11 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 71DC7CB3; Mon, 23 Sep 2013 06:23:11 +0000 (UTC) (envelope-from jiashiun@gmail.com) Received: from mail-ve0-x233.google.com (mail-ve0-x233.google.com [IPv6:2607:f8b0:400c:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 215A821A9; Mon, 23 Sep 2013 06:23:11 +0000 (UTC) Received: by mail-ve0-f179.google.com with SMTP id c14so1907260vea.38 for ; Sun, 22 Sep 2013 23:23:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=wEkIsMFCtxXIrwTeLqXlyqA1Z/cuqqXqk8vEr9pC038=; b=IL9OBphDIhrHBprUZYESgZXS96EIgoFdnHpWx/8u2C8gUYdn7IK2tBVcnPMZBVvpbK m15ky0Yljf2K88IGDfLlTj65RhJ0Lwb16GS1zMhejXkHUOXdP0bzNbNoxeIO+xLsq+WA rEgS0qxCln3Jf3UcgcbM9wUpSvodHD8vq69BBPyzAufNl7lavWQQcGxRQ7sU6vGoO51q Wdt8OMSpS3ZXqJm+q6gnbVsNNHPAGoGY1UlQHdqyYQ9g+EYQN7OxIQzpZl6NynmkJN6n ZL4rKWpvgubEjjsLrJLmQAwC/zF0Oc6p6gfaMB70nM+iHYvigFadhcn7lbFvhH1CoTQk CF2g== X-Received: by 10.220.237.208 with SMTP id kp16mr20500450vcb.4.1379917390166; Sun, 22 Sep 2013 23:23:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.225.34 with HTTP; Sun, 22 Sep 2013 23:22:40 -0700 (PDT) In-Reply-To: References: From: Jia-Shiun Li Date: Mon, 23 Sep 2013 14:22:40 +0800 Message-ID: Subject: Re: What's the state of AF-4Kn support? To: Ravi Pokala Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, "freebsd-hardware@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 06:23:11 -0000 On Wed, Sep 18, 2013 at 10:49 PM, Ravi Pokala wrote: > > What you describe is the 'AF-512e' format - 4KB physical sectors > *emulating* 512B logical sectors. See [ > https://en.wikipedia.org/wiki/Advanced_Format#Advanced_Format_512e ; > http://www.idema.org/?page_id=2153 ]. With AF-512e, yes, the HDD firmware > does the read/modify/write for I/Os smaller than the physical sector size. > It is intended to be a low-cost/transitional format, allowing the HDD > vendors to get the advantages of 4KB physical sectors (better error > detection/correction algorithms, better areal density => lower cost) w/o > breaking compatibility with decades of firmware and software that expect > 512B logical sectors. > > What I'm asking about is AF-4kn - 4KB *logical* as well as physical > sectors. All the enterprise HDD vendors have told us is that AF-4Kn drives > expect data IO to be 4KB, and will reject smaller transfers. (*metadata* > IO - SMART, IDENTIFY_DEVICE, READ_LOG/WRITE_LOG, etc - will remain 512B.) > > Doing some more digging, I found this post from ivoras which I missed the > first time around [ > http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector-drives.html ]; > that tends to support my initial assessment - filesystem stuff should Just > Work[tm] - plus adds the detail that direct drive I/O (the example he > gives is trying to `dd' 10 bytes) will be rejected because it is smaller > than the raw-device access granularity. I've also looked at 'ata_da.c' and > see that adaregister() looks at both quirks and IDENTIFY_DEVICE data to > determine the logical block size. > > So, that leaves the bootstrap code as the remaining question-mark. Does > anyone what AF-4Kn support looks like there? > CC -hackers. Thanks for the clarification. Is there any 4Kn HDDs shopping now? I am not aware of any. BTW I believe UFS and ZFS have proper design for 4K-sectors, but FreeBSD needs some ecosystem connections to get samples early to test, incorporate supports and validate for it. Or we will need to wait until it appears on market and someone got caught into some kind of bugs. Regards, Jia-Shiun. From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 08:14:14 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1DDED85B; Mon, 23 Sep 2013 08:14:14 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-bk0-x229.google.com (mail-bk0-x229.google.com [IPv6:2a00:1450:4008:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7C4492705; Mon, 23 Sep 2013 08:14:13 +0000 (UTC) Received: by mail-bk0-f41.google.com with SMTP id na10so1099680bkb.28 for ; Mon, 23 Sep 2013 01:14:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=reply-to:from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=I4+ffUWY/nGbeiGVQmmRbHthV8hYdpRPz82yS4UcMO8=; b=cIj8bErT1fh9B4HxJseLOA9fssruqGszaP4n+pSWEWiRRYUQXvhChqBmE6BmbGVqGs LasI3HrjRWq4NK3D9Avk45uO65pt500RlxZYjnxzLkdm8aYvOtM1Q0ggExeOfrxBZ0wA Q3R95GElxr1TQkU8Z4O0GG0s6s6U2I5lKvzNwHDaQCZ4R7jcja6n52ecsMxc0YTIo+bQ zcB0UUCJyU6BtAL0tqv0KGlg2kJ0pZal+InyxNJqScrVer1qg2aSXZP+KAG94M0SLHcs 7Wn2qP9V05R4RdcVEFTk9UeAQfuGQOYyn8T34q5DSMmlKGjj1wc0oZoSquIMpd3ddHLq 7GHQ== X-Received: by 10.205.15.72 with SMTP id pt8mr16452362bkb.17.1379924051312; Mon, 23 Sep 2013 01:14:11 -0700 (PDT) Received: from rimwks1w7x64 ([2001:470:1f15:8e:c8ff:267:ab31:49e1]) by mx.google.com with ESMTPSA id jt14sm8176913bkb.0.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 23 Sep 2013 01:14:10 -0700 (PDT) From: rozhuk.im@gmail.com To: "'Jia-Shiun Li'" , "'Ravi Pokala'" References: In-Reply-To: Subject: RE: What's the state of AF-4Kn support? Date: Mon, 23 Sep 2013 12:13:58 +0400 Message-ID: <523ff852.4ee8cc0a.44b6.ffffacba@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Ac64JWd+qAS+eZxGTWGEQro0C6z95AADz35Q Content-Language: ru Cc: freebsd-hackers@freebsd.org, freebsd-hardware@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Rozhuk.IM@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 08:14:14 -0000 # install gpart create -s GPT ada1 gpart show gpart add -i 1 -t freebsd-boot -b 40 -s 512 ada1 gpart add -i 2 -t freebsd-ufs -b 552 -s ..... ada1 gpart bootcode -b /boot/pmbr ada1 gpart bootcode -p /boot/gptboot -i 1 ada1 # for data gpart create -s GPT ada1 gpart show gpart add -i 1 -t freebsd-ufs -b 40 -s ..... ada1 > -----Original Message----- > From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd- > hackers@freebsd.org] On Behalf Of Jia-Shiun Li > Sent: Monday, September 23, 2013 10:23 AM > To: Ravi Pokala > Cc: freebsd-hackers@freebsd.org; freebsd-hardware@freebsd.org > Subject: Re: What's the state of AF-4Kn support? > > On Wed, Sep 18, 2013 at 10:49 PM, Ravi Pokala > wrote: > > > > What you describe is the 'AF-512e' format - 4KB physical sectors > > *emulating* 512B logical sectors. See [ > > https://en.wikipedia.org/wiki/Advanced_Format#Advanced_Format_512e ; > > http://www.idema.org/?page_id=2153 ]. With AF-512e, yes, the HDD > > firmware does the read/modify/write for I/Os smaller than the > physical sector size. > > It is intended to be a low-cost/transitional format, allowing the HDD > > vendors to get the advantages of 4KB physical sectors (better error > > detection/correction algorithms, better areal density => lower cost) > > w/o breaking compatibility with decades of firmware and software that > > expect 512B logical sectors. > > > > What I'm asking about is AF-4kn - 4KB *logical* as well as physical > > sectors. All the enterprise HDD vendors have told us is that AF-4Kn > > drives expect data IO to be 4KB, and will reject smaller transfers. > > (*metadata* IO - SMART, IDENTIFY_DEVICE, READ_LOG/WRITE_LOG, etc - > > will remain 512B.) > > > > Doing some more digging, I found this post from ivoras which I missed > > the first time around [ > > http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector- > drives.htm > > l ]; that tends to support my initial assessment - filesystem stuff > > should Just Work[tm] - plus adds the detail that direct drive I/O > (the > > example he gives is trying to `dd' 10 bytes) will be rejected because > > it is smaller than the raw-device access granularity. I've also > looked > > at 'ata_da.c' and see that adaregister() looks at both quirks and > > IDENTIFY_DEVICE data to determine the logical block size. > > > > So, that leaves the bootstrap code as the remaining question-mark. > > Does anyone what AF-4Kn support looks like there? > > > > CC -hackers. > > Thanks for the clarification. Is there any 4Kn HDDs shopping now? I am > not aware of any. > > BTW I believe UFS and ZFS have proper design for 4K-sectors, but > FreeBSD needs some ecosystem connections to get samples early to test, > incorporate supports and validate for it. Or we will need to wait until > it appears on market and someone got caught into some kind of bugs. > > Regards, > Jia-Shiun. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers- > unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 09:58:35 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DED151DB for ; Mon, 23 Sep 2013 09:58:35 +0000 (UTC) (envelope-from klaus@mx.7he.at) Received: from smtp-01.sil.at (smtp-01.sil.at [78.142.186.24]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9FC032CB4 for ; Mon, 23 Sep 2013 09:58:35 +0000 (UTC) Received: from mx.7he.at ([86.59.13.138]) by smtp-01.sil.at with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1VO2ed-00040A-9i for freebsd-hackers@freebsd.org; Mon, 23 Sep 2013 11:42:07 +0200 Received: from mx.7he.at (mx.7he.at [86.59.13.138]) by mx.7he.at (8.14.3/8.14.3) with ESMTP id r8N9foQV097622 for ; Mon, 23 Sep 2013 09:41:50 GMT (envelope-from klaus@mx.7he.at) Received: (from klaus@localhost) by mx.7he.at (8.14.3/8.14.3/Submit) id r8N9fofl097621 for freebsd-hackers@freebsd.org; Mon, 23 Sep 2013 09:41:50 GMT (envelope-from klaus) Date: Mon, 23 Sep 2013 09:41:50 +0000 From: "Klaus P. Ohrhallinger" To: freebsd-hackers@freebsd.org Subject: VPS Project Message-ID: <20130923094150.GM12210@mx.7he.at> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Political-Attitude: Anarchistic X-PGP-Key: http://www.klaus.priv.at/klaus.asc X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on mx.7he.at X-Scan-Signature: 3eda1eb18690ac26d5869843136654cc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 09:58:35 -0000 Hello; My virtualization project (http://www.7he.at/freebsd/vps/) has its project branch on svn.freebsd.org since a few months now. Since then there was not much progress due to lack of time. Now I am sitting on code updates that I can't commit myself. What is necessary to get commit right to /projects/vps ? Also it would be great if some FreeBSD developers could review parts of the code. So far, most feedback came from non-developers. Klaus -- * * * * * * http://www.ipv6actnow.org/ * * * * * * From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 11:37:46 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2200466A; Mon, 23 Sep 2013 11:37:46 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id D0D8624F2; Mon, 23 Sep 2013 11:37:45 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1VO4UW-0001lI-DE; Mon, 23 Sep 2013 15:39:48 +0400 Date: Mon, 23 Sep 2013 15:39:48 +0400 From: Slawa Olhovchenkov To: "Alexander V. Chernikov" Subject: Re: Network stack changes Message-ID: <20130923113948.GA5647@zxy.spb.ru> References: <521E41CB.30700@yandex-team.ru> <521E78B0.6080709@freebsd.org> <523F4BED.6000804@yandex-team.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <523F4BED.6000804@yandex-team.ru> User-Agent: Mutt/1.5.21 (2010-09-15) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-Mailman-Approved-At: Mon, 23 Sep 2013 11:59:30 +0000 Cc: adrian@freebsd.org, Andre Oppermann , freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org, luigi@freebsd.org, ae@FreeBSD.org, Gleb Smirnoff , FreeBSD Net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 11:37:46 -0000 On Sun, Sep 22, 2013 at 11:58:37PM +0400, Alexander V. Chernikov wrote: > I've found the paper I was talking about: > http://info.iet.unipi.it/~luigi/papers/20120601-dxr.pdf > > It claims that our radix is able to do 6MPPS/core and it does not scale > with number of cores. Our radix is bugly and don't work corretly. From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 13:59:00 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C9B328A3 for ; Mon, 23 Sep 2013 13:59:00 +0000 (UTC) (envelope-from havrik@bmail.ru) Received: from sovam.com (mail.bmail.ru [194.67.1.203]) by mx1.freebsd.org (Postfix) with ESMTP id 557B92F15 for ; Mon, 23 Sep 2013 13:58:59 +0000 (UTC) X-DSPAM-Result: Innocent X-DSPAM-Processed: Mon Sep 23 17:53:52 2013 X-DSPAM-Confidence: 0.9899 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 524047f0322131911969189 Received: from [93.81.249.2] (account havrik@bmail.ru) by mail-be03.sovam.com (CommuniGate Pro WEBUSER 5.2.20) with HTTP id 14814105 for freebsd-hackers@freebsd.org; Mon, 23 Sep 2013 17:53:51 +0400 From: Subject: 9.1-RELEASE-p7 devfs_ruleset To: FreeBSD Hackers X-Mailer: CommuniGate Pro WebUser v5.2.20 Date: Mon, 23 Sep 2013 17:53:51 +0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8;format="flowed" Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 13:59:00 -0000 # uname -r 9.1-RELEASE-p7 # mount -t devfs devfs /var/jail/test/dev/ # devfs -m /var/jail/test/dev/ rule showsets # devfs -m /var/jail/test/dev/ ruleset 4 # devfs -m /var/jail/test/dev/ rule applyset # devfs -m /var/jail/test/dev/ rule showsets 4 # ls /var/jail/test/dev/ acpi audit cpuctl1 devstat fido kbd0 midistat pci stdin ttyv2 ttyva ugen4.2 usbctl ad4 bpf crypto dgdb fw0 kbd1 mixer0 psm0 stdout ttyv3 ttyvb ugen5.1 xpt0 ad4p1 bpf0 ctty dsp0.0 fw0.0 kbdmux0 null ptmx sysmouse ttyv4 ugen0.1 ugen5.2 zero ad4p2 bpsm0 cuau0 dsp0.1 fwmem0 klog nvidia0 pts/ ttyu0 ttyv5 ugen0.2 ugen5.3 zfs ada0 cd0 cuau0.init dsp0.2 fwmem0.0 kmem nvidiactl random ttyu0.init ttyv6 ugen1.1 ugen6.1 zvol/ ada0p1 console cuau0.lock dsp0.3 geom.ctl led/ pass0 rtc ttyu0.lock ttyv7 ugen2.1 ums0 ada0p2 consolectl dcons dsp0.4 gptid/ mdctl pass1 sndstat ttyv0 ttyv8 ugen3.1 urandom atkbd0 cpuctl0 devctl fd/ io mem pccard0.cis stderr ttyv1 ttyv9 ugen4.1 usb/ PS: 9.1-RELEASE-p2 works correctly -- "Hi! I am a .signature virus! Copy me into your ~/.signature to help me spread!" From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 15:58:44 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D6EB2FC2; Mon, 23 Sep 2013 15:58:44 +0000 (UTC) (envelope-from rp_freebsd@mac.com) Received: from st11p00mm-asmtp002.mac.com (st11p00mm-asmtp002.mac.com [17.172.81.1]) by mx1.freebsd.org (Postfix) with ESMTP id AE5E226BD; Mon, 23 Sep 2013 15:58:44 +0000 (UTC) Received: from [192.168.1.3] (c-98-207-91-59.hsd1.ca.comcast.net [98.207.91.59]) by st11p00mm-asmtp002.mac.com (Oracle Communications Messaging Server 7u4-27.08(7.0.4.27.7) 64bit (built Aug 22 2013)) with ESMTPSA id <0MTL00MFW2X9L820@st11p00mm-asmtp002.mac.com>; Mon, 23 Sep 2013 14:58:38 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794,1.0.431,0.0.0000 definitions=2013-09-23_01:2013-09-22,2013-09-23,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1308280000 definitions=main-1309230071 User-Agent: Microsoft-MacOutlook/14.3.7.130812 Date: Mon, 23 Sep 2013 07:58:19 -0700 Subject: Re: What's the state of AF-4Kn support? From: Ravi Pokala Sender: Ravi Pokala To: Jia-Shiun Li Message-id: Thread-topic: What's the state of AF-4Kn support? In-reply-to: MIME-version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7bit X-Mailman-Approved-At: Mon, 23 Sep 2013 16:22:43 +0000 Cc: freebsd-hackers@freebsd.org, "freebsd-hardware@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 15:58:44 -0000 -----Original Message----- From: Jia-Shiun Li Date: Sunday, September 22, 2013 11:22 PM To: Ravi Pokala Cc: "freebsd-hardware@freebsd.org" , Subject: Re: What's the state of AF-4Kn support? >On Wed, Sep 18, 2013 at 10:49 PM, Ravi Pokala wrote: >> >>... > >CC -hackers. > >Thanks for the clarification. Is there any 4Kn HDDs shopping now? I am >not aware of any. Good question. I had the impression that some currently shipping drives were AF-4Kn, but spot-checking some of the drives listed in src/cam/ata/ata_da.c::ada_quirk_table[] against their datasheets, suggests that they're AF-512e. So, their being flagged w/ ADA_Q_4K is "just" a performance optimization. >BTW I believe UFS and ZFS have proper design for 4K-sectors, but FreeBSD >needs some ecosystem connections to get samples early to test, >incorporate supports and validate for it. Or we will need to wait until >it appears on market and someone got caught into some kind of bugs. Yeah, based on my reading of the code, it looks like the ATACAM layer and higher (GEOM, filesystems) take the physical block size into account. That just leaves the bootstrap code. Now that I've taken a second look, it seems as though at least 'pmbr' only works in terms of 512 bytes. :-( WRT to getting samples, it may be possible to share some w/ appropriate members of the community; who would those people be? >Regards, >Jia-Shiun. Thanks, rp From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 16:05:21 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0FC3333F; Mon, 23 Sep 2013 16:05:21 +0000 (UTC) (envelope-from rp_freebsd@mac.com) Received: from st11p00mm-asmtp003.mac.com (st11p00mm-asmtp003.mac.com [17.172.81.2]) by mx1.freebsd.org (Postfix) with ESMTP id DAE582749; Mon, 23 Sep 2013 16:05:20 +0000 (UTC) Received: from [192.168.1.3] (c-98-207-91-59.hsd1.ca.comcast.net [98.207.91.59]) by st11p00mm-asmtp003.mac.com (Oracle Communications Messaging Server 7u4-27.08(7.0.4.27.7) 64bit (built Aug 22 2013)) with ESMTPSA id <0MTL00KIQ3797050@st11p00mm-asmtp003.mac.com>; Mon, 23 Sep 2013 15:04:30 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794,1.0.431,0.0.0000 definitions=2013-09-23_01:2013-09-22,2013-09-23,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=2 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1308280000 definitions=main-1309230073 User-Agent: Microsoft-MacOutlook/14.3.7.130812 Date: Mon, 23 Sep 2013 08:04:19 -0700 Subject: Re: What's the state of AF-4Kn support? From: Ravi Pokala Sender: Ravi Pokala To: Rozhuk.IM@gmail.com, 'Jia-Shiun Li' Message-id: Thread-topic: What's the state of AF-4Kn support? In-reply-to: <523ff852.4ee8cc0a.44b6.ffffacba@mx.google.com> MIME-version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7bit X-Mailman-Approved-At: Mon, 23 Sep 2013 16:35:39 +0000 Cc: freebsd-hackers@freebsd.org, freebsd-hardware@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 16:05:21 -0000 -----Original Message----- From: Reply-To: Date: Monday, September 23, 2013 1:13 AM To: 'Jia-Shiun Li' , Ravi Pokala Cc: , Subject: RE: What's the state of AF-4Kn support? ># install >gpart create -s GPT ada1 >gpart show >gpart add -i 1 -t freebsd-boot -b 40 -s 512 ada1 >gpart add -i 2 -t freebsd-ufs -b 552 -s ..... ada1 >gpart bootcode -b /boot/pmbr ada1 >gpart bootcode -p /boot/gptboot -i 1 ada1 > ># for data >gpart create -s GPT ada1 >gpart show >gpart add -i 1 -t freebsd-ufs -b 40 -s ..... ada1 Thanks Rozhuk, As it happens, I know how to create a GPT table and make sure the partitions are aligned. That's certainly a necessary step for getting good performance on AF-512e drives, but it doesn't have much to do with booting from an AF-4Kn drive. (Actually, since GPT is defined in terms of physical sectors, we can actually *stop* playing silly alignment games with AF-4Kn, since it is inherently aligned.) --rp >> -----Original Message----- >> From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd- >> hackers@freebsd.org] On Behalf Of Jia-Shiun Li >> Sent: Monday, September 23, 2013 10:23 AM >> To: Ravi Pokala >> Cc: freebsd-hackers@freebsd.org; freebsd-hardware@freebsd.org >> Subject: Re: What's the state of AF-4Kn support? >> >> On Wed, Sep 18, 2013 at 10:49 PM, Ravi Pokala >> wrote: >> > >> > What you describe is the 'AF-512e' format - 4KB physical sectors >> > *emulating* 512B logical sectors. See [ >> > https://en.wikipedia.org/wiki/Advanced_Format#Advanced_Format_512e ; >> > http://www.idema.org/?page_id=2153 ]. With AF-512e, yes, the HDD >> > firmware does the read/modify/write for I/Os smaller than the >> physical sector size. >> > It is intended to be a low-cost/transitional format, allowing the HDD >> > vendors to get the advantages of 4KB physical sectors (better error >> > detection/correction algorithms, better areal density => lower cost) >> > w/o breaking compatibility with decades of firmware and software that >> > expect 512B logical sectors. >> > >> > What I'm asking about is AF-4kn - 4KB *logical* as well as physical >> > sectors. All the enterprise HDD vendors have told us is that AF-4Kn >> > drives expect data IO to be 4KB, and will reject smaller transfers. >> > (*metadata* IO - SMART, IDENTIFY_DEVICE, READ_LOG/WRITE_LOG, etc - >> > will remain 512B.) >> > >> > Doing some more digging, I found this post from ivoras which I missed >> > the first time around [ >> > http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector- >> drives.htm >> > l ]; that tends to support my initial assessment - filesystem stuff >> > should Just Work[tm] - plus adds the detail that direct drive I/O >> (the >> > example he gives is trying to `dd' 10 bytes) will be rejected because >> > it is smaller than the raw-device access granularity. I've also >> looked >> > at 'ata_da.c' and see that adaregister() looks at both quirks and >> > IDENTIFY_DEVICE data to determine the logical block size. >> > >> > So, that leaves the bootstrap code as the remaining question-mark. >> > Does anyone what AF-4Kn support looks like there? >> > >> >> CC -hackers. >> >> Thanks for the clarification. Is there any 4Kn HDDs shopping now? I am >> not aware of any. >> >> BTW I believe UFS and ZFS have proper design for 4K-sectors, but >> FreeBSD needs some ecosystem connections to get samples early to test, >> incorporate supports and validate for it. Or we will need to wait until >> it appears on market and someone got caught into some kind of bugs. >> >> Regards, >> Jia-Shiun. >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers- >> unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 16:38:53 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 02CFACCD; Mon, 23 Sep 2013 16:38:53 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email2.allantgroup.com (email2.emsphone.com [199.67.51.116]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AB92529B4; Mon, 23 Sep 2013 16:38:51 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [172.17.17.101]) by email2.allantgroup.com (8.14.5/8.14.5) with ESMTP id r8NGciv1041571 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 23 Sep 2013 11:38:44 -0500 (CDT) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.7/8.14.6) with ESMTP id r8NGcifo033918 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 23 Sep 2013 11:38:44 -0500 (CDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.7/8.14.7/Submit) id r8NGciBP033917; Mon, 23 Sep 2013 11:38:44 -0500 (CDT) (envelope-from dan) Date: Mon, 23 Sep 2013 11:38:44 -0500 From: Dan Nelson To: Jia-Shiun Li Subject: Re: What's the state of AF-4Kn support? Message-ID: <20130923163844.GE97298@dan.emsphone.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 9.1-STABLE User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.97.8 at email2.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (email2.allantgroup.com [172.17.19.78]); Mon, 23 Sep 2013 11:38:44 -0500 (CDT) X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, RP_MATCHES_RCVD autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on email2.allantgroup.com X-Scanned-By: MIMEDefang 2.73 Cc: Ravi Pokala , freebsd-hackers@freebsd.org, "freebsd-hardware@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 16:38:53 -0000 In the last episode (Sep 23), Jia-Shiun Li said: > On Wed, Sep 18, 2013 at 10:49 PM, Ravi Pokala wrote: > > What I'm asking about is AF-4kn - 4KB *logical* as well as physical > > sectors. All the enterprise HDD vendors have told us is that AF-4Kn drives > > expect data IO to be 4KB, and will reject smaller transfers. (*metadata* > > IO - SMART, IDENTIFY_DEVICE, READ_LOG/WRITE_LOG, etc - will remain 512B.) > > > > Doing some more digging, I found this post from ivoras which I missed the > > first time around [ > > http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector-drives.html ]; > > that tends to support my initial assessment - filesystem stuff should Just > > Work[tm] - plus adds the detail that direct drive I/O (the example he > > gives is trying to `dd' 10 bytes) will be rejected because it is smaller > > than the raw-device access granularity. I've also looked at 'ata_da.c' and > > see that adaregister() looks at both quirks and IDENTIFY_DEVICE data to > > determine the logical block size. > > > > So, that leaves the bootstrap code as the remaining question-mark. Does > > anyone what AF-4Kn support looks like there? > > > > CC -hackers. > > Thanks for the clarification. Is there any 4Kn HDDs shopping now? I am > not aware of any. I don't think there are any yet, but some SATA->USB drive enclosures will present a 4Kn drive to the host if the physical drive is 512e. The Seagate Backup Plus does this at least. It lets you continue to use MBR-based partitioning and still access all of a 4TB disk. Unfortunately, since both GPT and MBR work off of block offsets, partitions created in one mode won't work in the other, so you can't just swap a disk in and out of the enclosure without (carefully) repartitioning. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 17:16:13 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BF913A54; Mon, 23 Sep 2013 17:16:13 +0000 (UTC) (envelope-from S.Kuzminsky@F5.com) Received: from mail.f5.com (mail.f5.com [208.85.209.139]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 763112BCC; Mon, 23 Sep 2013 17:16:13 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.90,963,1371081600"; d="scan'208";a="82507357" Received: from unknown (HELO exchmail.f5net.com) ([192.168.10.235]) by seamgw02.olympus.f5net.com with ESMTP; 23 Sep 2013 17:16:06 +0000 Received: from SEAEMBX01.olympus.F5Net.com ([fe80::3440:4256:38f6:d3a0]) by seaecas02.olympus.F5Net.com ([::1]) with mapi id 14.03.0158.001; Mon, 23 Sep 2013 10:16:06 -0700 From: Sebastian Kuzminsky To: Cedric Blancher Subject: Re: About Transparent Superpages and Non-transparent superapges Thread-Topic: About Transparent Superpages and Non-transparent superapges Thread-Index: AQHOtIlQ/xFJtaAUVEqnjcRRWECf45nN+7CAgAB+KYCAALxWAIAApIIAgAQywIA= Date: Mon, 23 Sep 2013 17:16:06 +0000 Message-ID: References: <1379520488.49964.YahooMailNeo@web193502.mail.sg3.yahoo.com> <22E7E628-E997-4B64-B229-92E425D85084@f5.com> <1379649991.82562.YahooMailNeo@web193502.mail.sg3.yahoo.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.16.236] Content-Type: text/plain; charset="iso-8859-1" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Sebastian Kuzminsky , Patrick Dung , "ivoras@freebsd.org" , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 17:16:13 -0000 On Sep 20, 2013, at 19:09 , Cedric Blancher wrote: > On 20 September 2013 17:20, Sebastian Kuzminsky wrot= e: >> On Sep 19, 2013, at 22:06 , Patrick Dung wrote: >>=20 >>>> We at Line Rate (now F5) are developing support for 1 Gig superpages o= n amd64. We're basing our work on 9.1.0 for now. >>>>=20 >>>> An early preview is available here: >>>>=20 >>>> https://github.com/Seb-LineRate/freebsd/tree/freebsd-9.1.0-1gig-pages-= NOT-READY-2 >>>=20 >>> That is cool. >>>=20 >>> What type of applications can take advantage of the 1Gb page size? >>> And is it transparent? Or applications need to be modified? >>=20 >> It's transparent for the kernel: all of UMA and kmem_malloc()/kmem_free(= ) is backed by 1 gig superpages. >>=20 >> It's not transparent for userspace: applications need to pass a new flag= to mmap() to get 1 gig pages. >=20 > That may be the wrong approach. What happens if x86 gets more > huge/largepage sizes like SPARC does (hint: Sign NDA with Intel and > AMD and get surprised, and then allocate 16 more bits for mmap() if > you wish to stick with your approach)? For example SPARC64 does 8k, > 64k, 512k, 4M, 32M, 256M, 2GB and 256GB pages (actual page sizes > differ from MMU to MMU implementation, and can be probed via pagesize > -a). >=20 > A much better option would be to follow the Solaris API which has APIs > to enumerate the available page sizes, and then set it either for > heap, stack or a given address range (the last one is used to use > largepages for file I/O via mmap()). I discussed this API with Alan Cox at the FreeBSD Developers' Summit in Ott= awa in May 2013, and he agreed that this was a good place to start. The ma= in concern he had at the time was that the buddy allocator I implemented in= kmem_malloc() might need improvements. The API you describe sounds more flexible and possibly preferable, and I'm = open to discussing it as an enhancement. --=20 Sebastian Kuzminsky From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 21:24:49 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A224EC85; Mon, 23 Sep 2013 21:24:49 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x236.google.com (mail-wg0-x236.google.com [IPv6:2a00:1450:400c:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 16C672AD4; Mon, 23 Sep 2013 21:24:48 +0000 (UTC) Received: by mail-wg0-f54.google.com with SMTP id m15so3612695wgh.21 for ; Mon, 23 Sep 2013 14:24:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=MW0Od4HXzn+UXCNAJvj90awziJwZGR0O4HGWnNXpEYI=; b=nF0AotuLglv0ZwryHJJltnx/LYTh5Spj/EdMhFiJk+PrUfZN9r2Y61dXn7dsaMkDyJ tEEnf/Of0yC+B80ETYL/zTi3xV7oYwPtL4L0GlqUIBe3YFtmT+/kb0xzfhXW+Srz52Jx idAO6VXbmOKmtNF/AlLELsDf7R3F6XO86+G5DrtY8abxy87CzAvJAGzmwsdWU73Dg9zy zgnPQCZ0dQA3gs+W4YTU1fGj/aqORJt8JYhrIvfvsolFUoHAxiG6MKCNdV4/4ykCaSnp N0bReVsAOI4yFZ851xQK6AbCjiyRRwqGm0lrxACJdajxOaeS7f4kP6njENZaJoAnnpeV IF7g== MIME-Version: 1.0 X-Received: by 10.194.93.105 with SMTP id ct9mr19297600wjb.6.1379971487421; Mon, 23 Sep 2013 14:24:47 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.73.133 with HTTP; Mon, 23 Sep 2013 14:24:47 -0700 (PDT) In-Reply-To: References: <1379520488.49964.YahooMailNeo@web193502.mail.sg3.yahoo.com> <22E7E628-E997-4B64-B229-92E425D85084@f5.com> <1379649991.82562.YahooMailNeo@web193502.mail.sg3.yahoo.com> Date: Mon, 23 Sep 2013 14:24:47 -0700 X-Google-Sender-Auth: rq1IazfSjiyOg_vdiIgKTfvJr5o Message-ID: Subject: Re: About Transparent Superpages and Non-transparent superapges From: Adrian Chadd To: Sebastian Kuzminsky Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Patrick Dung , "freebsd-hackers@freebsd.org" , "ivoras@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 21:24:49 -0000 On 20 September 2013 08:20, Sebastian Kuzminsky wrote: > It's transparent for the kernel: all of UMA and kmem_malloc()/kmem_free() > is backed by 1 gig superpages. > .. not entirely true, as I've found out at work. :( -adrian From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 21:30:04 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 873B7FA1; Mon, 23 Sep 2013 21:30:04 +0000 (UTC) (envelope-from S.Kuzminsky@F5.com) Received: from mail.f5.com (mail.f5.com [208.85.209.139]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 538722B30; Mon, 23 Sep 2013 21:30:04 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.90,965,1371081600"; d="scan'208";a="82531724" Received: from unknown (HELO exchmail.f5net.com) ([192.168.10.235]) by seamgw02.olympus.f5net.com with ESMTP; 23 Sep 2013 21:30:03 +0000 Received: from SEAEMBX01.olympus.F5Net.com ([fe80::3440:4256:38f6:d3a0]) by SEAECAS03.olympus.F5Net.com ([::1]) with mapi id 14.03.0158.001; Mon, 23 Sep 2013 14:30:03 -0700 From: Sebastian Kuzminsky To: Adrian Chadd Subject: Re: About Transparent Superpages and Non-transparent superapges Thread-Topic: About Transparent Superpages and Non-transparent superapges Thread-Index: AQHOtIlQ/xFJtaAUVEqnjcRRWECf45nN+7CAgAB+KYCAALxWAIAFHL+AgAABdwA= Date: Mon, 23 Sep 2013 21:30:03 +0000 Message-ID: References: <1379520488.49964.YahooMailNeo@web193502.mail.sg3.yahoo.com> <22E7E628-E997-4B64-B229-92E425D85084@f5.com> <1379649991.82562.YahooMailNeo@web193502.mail.sg3.yahoo.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.16.250] Content-Type: text/plain; charset="iso-8859-1" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Sebastian Kuzminsky , Patrick Dung , "ivoras@freebsd.org" , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 21:30:04 -0000 On Sep 23, 2013, at 15:24 , Adrian Chadd wrote: > On 20 September 2013 08:20, Sebastian Kuzminsky wrot= e: > =20 > It's transparent for the kernel: all of UMA and kmem_malloc()/kmem_free()= is backed by 1 gig superpages. >=20 > .. not entirely true, as I've found out at work. :( Can you expand on this, Adrian? Did you compile & boot the github branch i pointed to, and run in to a situ= ation where kmem_malloc() returned memory not backed by 1 gig pages, on har= dware that supports it? --=20 Sebastian Kuzminsky From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 23 22:46:47 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8EB6AE72; Mon, 23 Sep 2013 22:46:47 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4269C2F8F; Mon, 23 Sep 2013 22:46:47 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id x10so3809582pdj.29 for ; Mon, 23 Sep 2013 15:46:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=UjilBwU9OcaYlukiK4g1agmh0oG9lXC8IZz6YH7pBlg=; b=zNd2nK4fBP8S1qNkzhLLiAaFq3C4x9cs2l/LTZGD8uliIScxephPgtW+7SCmsbDgyP zPOwbSas1L4JqJYHFeXl+xDyHGgcKGQxW90RfR95vFmZzoqsZPL6IkW4nE98w++cfJsZ +e6CJwB7dPIYAg0V1byVyhshHzsVmidAdwxDXn3Aij1tX5RjR+b7zexAkbHtjh1aDB/L FafyXOjh/rDms1JVnjmCUv9t2V6LedW+gx722WWwMGk7TiOi77IdH6DE5B9lYEfwZ4g7 trkHotGxZ1WWlEq6SF2TtwsgUIzr8ghc2hO73/neyiPvynvGoPaLlM/NHjuRcQjZss2Q lv5A== MIME-Version: 1.0 X-Received: by 10.68.11.103 with SMTP id p7mr3431565pbb.84.1379976406783; Mon, 23 Sep 2013 15:46:46 -0700 (PDT) Received: by 10.70.30.98 with HTTP; Mon, 23 Sep 2013 15:46:46 -0700 (PDT) In-Reply-To: <523F4F14.9090404@yandex-team.ru> References: <521E41CB.30700@yandex-team.ru> <523F4F14.9090404@yandex-team.ru> Date: Tue, 24 Sep 2013 01:46:46 +0300 Message-ID: Subject: Re: Network stack changes From: Sami Halabi To: "Alexander V. Chernikov" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Adrian Chadd , Andre Oppermann , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" , Luigi Rizzo , "Andrey V. Elsukov" , FreeBSD Net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 22:46:47 -0000 Hi, > http://info.iet.unipi.it/~**luigi/papers/20120601-dxr.pdf > http://www.nxlab.fer.hr/dxr/**stable_8_20120824.diff I've tried the diff in 10-current, applied cleanly but had errors compiling new kernel... is there any work to make it work? i'd love to test it. Sami On Sun, Sep 22, 2013 at 11:12 PM, Alexander V. Chernikov < melifaro@yandex-team.ru> wrote: > On 29.08.2013 15:49, Adrian Chadd wrote: > >> Hi, >> > Hello Adrian! > I'm very sorry for the looong reply. > > > >> There's a lot of good stuff to review here, thanks! >> >> Yes, the ixgbe RX lock needs to die in a fire. It's kinda pointless to >> keep locking things like that on a per-packet basis. We should be able to >> do this in a cleaner way - we can defer RX into a CPU pinned taskqueue and >> convert the interrupt handler to a fast handler that just schedules that >> taskqueue. We can ignore the ithread entirely here. >> >> What do you think? >> > Well, it sounds good :) But performance numbers and Jack opinion is more > important :) > > Are you going to Malta? > > >> Totally pie in the sky handwaving at this point: >> >> * create an array of mbuf pointers for completed mbufs; >> * populate the mbuf array; >> * pass the array up to ether_demux(). >> >> For vlan handling, it may end up populating its own list of mbufs to push >> up to ether_demux(). So maybe we should extend the API to have a bitmap of >> packets to actually handle from the array, so we can pass up a larger array >> of mbufs, note which ones are for the destination and then the upcall can >> mark which frames its consumed. >> >> I specifically wonder how much work/benefit we may see by doing: >> >> * batching packets into lists so various steps can batch process things >> rather than run to completion; >> * batching the processing of a list of frames under a single lock >> instance - eg, if the forwarding code could do the forwarding lookup for >> 'n' packets under a single lock, then pass that list of frames up to >> inet_pfil_hook() to do the work under one lock, etc, etc. >> > I'm thinking the same way, but we're stuck with 'forwarding lookup' due to > problem with egress interface pointer, as I mention earlier. However it is > interesting to see how much it helps, regardless of locking. > > Currently I'm thinking that we should try to change radix to something > different (it seems that it can be checked fast) and see what happened. > Luigi's performance numbers for our radix are too awful, and there is a > patch implementing alternative trie: > http://info.iet.unipi.it/~**luigi/papers/20120601-dxr.pdf > http://www.nxlab.fer.hr/dxr/**stable_8_20120824.diff > > > > >> Here, the processing would look less like "grab lock and process to >> completion" and more like "mark and sweep" - ie, we have a list of frames >> that we mark as needing processing and mark as having been processed at >> each layer, so we know where to next dispatch them. >> >> I still have some tool coding to do with PMC before I even think about >> tinkering with this as I'd like to measure stuff like per-packet latency as >> well as top-level processing overhead (ie, CPU_CLK_UNHALTED.THREAD_P / >> lagg0 TX bytes/pkts, RX bytes/pkts, NIC interrupts on that core, etc.) >> > That will be great to see! > >> >> Thanks, >> >> >> >> -adrian >> >> > ______________________________**_________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org > " > -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert On Sun, Sep 22, 2013 at 11:12 PM, Alexander V. Chernikov < melifaro@yandex-team.ru> wrote: > On 29.08.2013 15:49, Adrian Chadd wrote: > >> Hi, >> > Hello Adrian! > I'm very sorry for the looong reply. > > > >> There's a lot of good stuff to review here, thanks! >> >> Yes, the ixgbe RX lock needs to die in a fire. It's kinda pointless to >> keep locking things like that on a per-packet basis. We should be able to >> do this in a cleaner way - we can defer RX into a CPU pinned taskqueue and >> convert the interrupt handler to a fast handler that just schedules that >> taskqueue. We can ignore the ithread entirely here. >> >> What do you think? >> > Well, it sounds good :) But performance numbers and Jack opinion is more > important :) > > Are you going to Malta? > > >> Totally pie in the sky handwaving at this point: >> >> * create an array of mbuf pointers for completed mbufs; >> * populate the mbuf array; >> * pass the array up to ether_demux(). >> >> For vlan handling, it may end up populating its own list of mbufs to push >> up to ether_demux(). So maybe we should extend the API to have a bitmap of >> packets to actually handle from the array, so we can pass up a larger array >> of mbufs, note which ones are for the destination and then the upcall can >> mark which frames its consumed. >> >> I specifically wonder how much work/benefit we may see by doing: >> >> * batching packets into lists so various steps can batch process things >> rather than run to completion; >> * batching the processing of a list of frames under a single lock >> instance - eg, if the forwarding code could do the forwarding lookup for >> 'n' packets under a single lock, then pass that list of frames up to >> inet_pfil_hook() to do the work under one lock, etc, etc. >> > I'm thinking the same way, but we're stuck with 'forwarding lookup' due to > problem with egress interface pointer, as I mention earlier. However it is > interesting to see how much it helps, regardless of locking. > > Currently I'm thinking that we should try to change radix to something > different (it seems that it can be checked fast) and see what happened. > Luigi's performance numbers for our radix are too awful, and there is a > patch implementing alternative trie: > http://info.iet.unipi.it/~**luigi/papers/20120601-dxr.pdf > http://www.nxlab.fer.hr/dxr/**stable_8_20120824.diff > > > > >> Here, the processing would look less like "grab lock and process to >> completion" and more like "mark and sweep" - ie, we have a list of frames >> that we mark as needing processing and mark as having been processed at >> each layer, so we know where to next dispatch them. >> >> I still have some tool coding to do with PMC before I even think about >> tinkering with this as I'd like to measure stuff like per-packet latency as >> well as top-level processing overhead (ie, CPU_CLK_UNHALTED.THREAD_P / >> lagg0 TX bytes/pkts, RX bytes/pkts, NIC interrupts on that core, etc.) >> > That will be great to see! > >> >> Thanks, >> >> >> >> -adrian >> >> > ______________________________**_________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org > " > -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 24 00:16:47 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2B6B69AC; Tue, 24 Sep 2013 00:16:47 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 85BA823A4; Tue, 24 Sep 2013 00:16:46 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id k14so3766510wgh.1 for ; Mon, 23 Sep 2013 17:16:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=LhM+3pPuO9en/vqO7xzKF2aEwHdfEVXtULHq5TNMgcY=; b=qsuTf5YlAbZfb7vcsGQeeo/arCTyQI/lGKVsyhVtH9cSfq9ka7MM3h7V2eeRt2rk67 17aovFoiK8hvJmvi0pAJ6PLfkBAnHThx+xutpTIHdapgxZIL4b3ZvIk5vJATSLl3DBF3 MwKtf4pxOJ96WeAJvQ1Rwub8a8NA3Q3/NXuH9dbWvQpulwzMOTEfb0nIuzDyi7/3dMC9 mSSETFqU1KQBy13fX+FMdtPTGkG4OiX7To79crMWtTBxbI5FtZKncbzaviqOl+VGmK5+ JYYsgeR08gxMFZ2hBE+cntMRouu2uomN4/IlBrY1E10RiL/U+eHXCAYWEbL+JgZYQlT7 +PBw== MIME-Version: 1.0 X-Received: by 10.180.10.136 with SMTP id i8mr15626450wib.46.1379981804885; Mon, 23 Sep 2013 17:16:44 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.73.133 with HTTP; Mon, 23 Sep 2013 17:16:44 -0700 (PDT) In-Reply-To: References: <1379520488.49964.YahooMailNeo@web193502.mail.sg3.yahoo.com> <22E7E628-E997-4B64-B229-92E425D85084@f5.com> <1379649991.82562.YahooMailNeo@web193502.mail.sg3.yahoo.com> Date: Mon, 23 Sep 2013 17:16:44 -0700 X-Google-Sender-Auth: tOC-hEMLG2MhA2xnqB-nbxZ6YNA Message-ID: Subject: Re: About Transparent Superpages and Non-transparent superapges From: Adrian Chadd To: Sebastian Kuzminsky Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Patrick Dung , "freebsd-hackers@freebsd.org" , "ivoras@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Sep 2013 00:16:47 -0000 On 23 September 2013 14:30, Sebastian Kuzminsky wrote: > On Sep 23, 2013, at 15:24 , Adrian Chadd wrote: > > > On 20 September 2013 08:20, Sebastian Kuzminsky > wrote: > > > > It's transparent for the kernel: all of UMA and > kmem_malloc()/kmem_free() is backed by 1 gig superpages. > > > > .. not entirely true, as I've found out at work. :( > > Can you expand on this, Adrian? > > Did you compile & boot the github branch i pointed to, and run in to a > situation where kmem_malloc() returned memory not backed by 1 gig pages, on > hardware that supports it? > > I haven't done that yet, sorry. So the direct map is backed by 1GB pages, except when it can't be: * first 1GB - because of the memory hole(s) * the 4th GB - because of the PCI IO hole(s) * the end of RAM - because of memory remapping so you don't lose hundreds of megabytes of RAM behind said memory/IO/ROM holes, the end of RAM isn't on a 1GB boundary. So, those regions seem to get mapped by smaller pages. I'm still tinkering with this; I'd like to hack things up to (a) get all the VM structures in the last gig of aligned RAM, so it falls inside a 1GB direct mapped page, and (b) prefer that 1GB page for kernel allocations, so things like mbufs, vm_page entries, etc all end up coming from the same 1GB direct map page. I _think_ I have an idea of what to do - I'll create a couple of 1GB sized freelists in the last two 1GB direct mapped regions at the end of RAM, then I'll hack up the vm_phys allocator to prefer allocating from those. The VM structures stuff is a bit more annoying - it gets allocated from the top of RAM early on during boot; so unless your machine has the last region of RAM fall exactly on a 1GB boundary, it'll be backed by 4k/2m pages. I tested this out by setting hw.physmem to force things to be rounded on a boundary and it helped for a while. Unfortunately the fact that everything else gets allocated from random places in physical memory meant that I'm thrashing the TLB cache - there's only 4 1GB slots on Sandy Bridge Xeon; and with 64gig of RAM I'm seeing a 10-12% miss load when serving lots of traffic from SSD (all with mbuf and vm structure allocations.) So, if I can remove that 10% of CPU cycles taken walking pages, I'll be happy. Note: I'm a newbie here in the physical mapping code. :-) -adrian From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 24 07:58:31 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2E99BDED; Tue, 24 Sep 2013 07:58:31 +0000 (UTC) (envelope-from zec@fer.hr) Received: from mail.fer.hr (mail.fer.hr [161.53.72.233]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5411428EC; Tue, 24 Sep 2013 07:58:29 +0000 (UTC) Received: from dyn10.nxlab.fer.hr (161.53.63.210) by MAIL.fer.hr (161.53.72.233) with Microsoft SMTP Server (TLS) id 14.2.342.3; Tue, 24 Sep 2013 09:57:14 +0200 From: Marko Zec To: Subject: Re: Network stack changes Date: Tue, 24 Sep 2013 09:58:05 +0200 User-Agent: KMail/1.9.10 References: <521E41CB.30700@yandex-team.ru> <523F4F14.9090404@yandex-team.ru> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-ID: <201309240958.06172.zec@fer.hr> X-Originating-IP: [161.53.63.210] Cc: "Alexander V. Chernikov" , Adrian Chadd , Andre Oppermann , FreeBSD Net , Luigi Rizzo , "Andrey V. Elsukov" , "freebsd-arch@freebsd.org" , Sami Halabi X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Sep 2013 07:58:31 -0000 On Tuesday 24 September 2013 00:46:46 Sami Halabi wrote: > Hi, > > > http://info.iet.unipi.it/~**luigi/papers/20120601-dxr.pdf >et.unipi.it/~luigi/papers/20120601-dxr.pdf> > > http://www.nxlab.fer.hr/dxr/**stable_8_20120824.diff >er.hr/dxr/stable_8_20120824.diff> > > I've tried the diff in 10-current, applied cleanly but had errors > compiling new kernel... is there any work to make it work? i'd love to > test it. Even if you'd make it compile on current, you could only run synthetic tests measuring lookup performance using streams of random keys, as outlined in the paper (btw. the paper at Luigi's site is an older draft, the final version with slightly revised benchmarks is available here: http://www.sigcomm.org/sites/default/files/ccr/papers/2012/October/2378956-2378961.pdf) I.e. the code only hooks into the routing API for testing purposes, but is completely disconnected from the forwarding path. We have a prototype in the works which combines DXR with Netmap in userspace and is capable of sustaining well above line rate forwarding with full-sized BGP views using Intel 10G cards on commodity multicore machines. The work was somewhat stalled during the summer but I plan to wrap it up and release the code until the end of this year. With recent advances in netmap it might also be feasible to merge DXR and netmap entirely inside the kernel but I've not explored that path yet... Marko > Sami > > > On Sun, Sep 22, 2013 at 11:12 PM, Alexander V. Chernikov < > > melifaro@yandex-team.ru> wrote: > > On 29.08.2013 15:49, Adrian Chadd wrote: > >> Hi, > > > > Hello Adrian! > > I'm very sorry for the looong reply. > > > >> There's a lot of good stuff to review here, thanks! > >> > >> Yes, the ixgbe RX lock needs to die in a fire. It's kinda pointless to > >> keep locking things like that on a per-packet basis. We should be able > >> to do this in a cleaner way - we can defer RX into a CPU pinned > >> taskqueue and convert the interrupt handler to a fast handler that > >> just schedules that taskqueue. We can ignore the ithread entirely > >> here. > >> > >> What do you think? > > > > Well, it sounds good :) But performance numbers and Jack opinion is > > more important :) > > > > Are you going to Malta? > > > >> Totally pie in the sky handwaving at this point: > >> > >> * create an array of mbuf pointers for completed mbufs; > >> * populate the mbuf array; > >> * pass the array up to ether_demux(). > >> > >> For vlan handling, it may end up populating its own list of mbufs to > >> push up to ether_demux(). So maybe we should extend the API to have a > >> bitmap of packets to actually handle from the array, so we can pass up > >> a larger array of mbufs, note which ones are for the destination and > >> then the upcall can mark which frames its consumed. > >> > >> I specifically wonder how much work/benefit we may see by doing: > >> > >> * batching packets into lists so various steps can batch process > >> things rather than run to completion; > >> * batching the processing of a list of frames under a single lock > >> instance - eg, if the forwarding code could do the forwarding lookup > >> for 'n' packets under a single lock, then pass that list of frames up > >> to inet_pfil_hook() to do the work under one lock, etc, etc. > > > > I'm thinking the same way, but we're stuck with 'forwarding lookup' due > > to problem with egress interface pointer, as I mention earlier. However > > it is interesting to see how much it helps, regardless of locking. > > > > Currently I'm thinking that we should try to change radix to something > > different (it seems that it can be checked fast) and see what happened. > > Luigi's performance numbers for our radix are too awful, and there is a > > patch implementing alternative trie: > > http://info.iet.unipi.it/~**luigi/papers/20120601-dxr.pdf >et.unipi.it/~luigi/papers/20120601-dxr.pdf> > > http://www.nxlab.fer.hr/dxr/**stable_8_20120824.diff >er.hr/dxr/stable_8_20120824.diff> > > > >> Here, the processing would look less like "grab lock and process to > >> completion" and more like "mark and sweep" - ie, we have a list of > >> frames that we mark as needing processing and mark as having been > >> processed at each layer, so we know where to next dispatch them. > >> > >> I still have some tool coding to do with PMC before I even think about > >> tinkering with this as I'd like to measure stuff like per-packet > >> latency as well as top-level processing overhead (ie, > >> CPU_CLK_UNHALTED.THREAD_P / lagg0 TX bytes/pkts, RX bytes/pkts, NIC > >> interrupts on that core, etc.) > > > > That will be great to see! > > > >> Thanks, > >> > >> > >> > >> -adrian > > > > ______________________________**_________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/**mailman/listinfo/freebsd-net >eebsd.org/mailman/listinfo/freebsd-net> To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@**freebsd.org >org> " > > -- > Sami Halabi > Information Systems Engineer > NMS Projects Expert > FreeBSD SysAdmin Expert > > > On Sun, Sep 22, 2013 at 11:12 PM, Alexander V. Chernikov < > > melifaro@yandex-team.ru> wrote: > > On 29.08.2013 15:49, Adrian Chadd wrote: > >> Hi, > > > > Hello Adrian! > > I'm very sorry for the looong reply. > > > >> There's a lot of good stuff to review here, thanks! > >> > >> Yes, the ixgbe RX lock needs to die in a fire. It's kinda pointless to > >> keep locking things like that on a per-packet basis. We should be able > >> to do this in a cleaner way - we can defer RX into a CPU pinned > >> taskqueue and convert the interrupt handler to a fast handler that > >> just schedules that taskqueue. We can ignore the ithread entirely > >> here. > >> > >> What do you think? > > > > Well, it sounds good :) But performance numbers and Jack opinion is > > more important :) > > > > Are you going to Malta? > > > >> Totally pie in the sky handwaving at this point: > >> > >> * create an array of mbuf pointers for completed mbufs; > >> * populate the mbuf array; > >> * pass the array up to ether_demux(). > >> > >> For vlan handling, it may end up populating its own list of mbufs to > >> push up to ether_demux(). So maybe we should extend the API to have a > >> bitmap of packets to actually handle from the array, so we can pass up > >> a larger array of mbufs, note which ones are for the destination and > >> then the upcall can mark which frames its consumed. > >> > >> I specifically wonder how much work/benefit we may see by doing: > >> > >> * batching packets into lists so various steps can batch process > >> things rather than run to completion; > >> * batching the processing of a list of frames under a single lock > >> instance - eg, if the forwarding code could do the forwarding lookup > >> for 'n' packets under a single lock, then pass that list of frames up > >> to inet_pfil_hook() to do the work under one lock, etc, etc. > > > > I'm thinking the same way, but we're stuck with 'forwarding lookup' due > > to problem with egress interface pointer, as I mention earlier. However > > it is interesting to see how much it helps, regardless of locking. > > > > Currently I'm thinking that we should try to change radix to something > > different (it seems that it can be checked fast) and see what happened. > > Luigi's performance numbers for our radix are too awful, and there is a > > patch implementing alternative trie: > > http://info.iet.unipi.it/~**luigi/papers/20120601-dxr.pdf >et.unipi.it/~luigi/papers/20120601-dxr.pdf> > > http://www.nxlab.fer.hr/dxr/**stable_8_20120824.diff >er.hr/dxr/stable_8_20120824.diff> > > > >> Here, the processing would look less like "grab lock and process to > >> completion" and more like "mark and sweep" - ie, we have a list of > >> frames that we mark as needing processing and mark as having been > >> processed at each layer, so we know where to next dispatch them. > >> > >> I still have some tool coding to do with PMC before I even think about > >> tinkering with this as I'd like to measure stuff like per-packet > >> latency as well as top-level processing overhead (ie, > >> CPU_CLK_UNHALTED.THREAD_P / lagg0 TX bytes/pkts, RX bytes/pkts, NIC > >> interrupts on that core, etc.) > > > > That will be great to see! > > > >> Thanks, > >> > >> > >> > >> -adrian > > > > ______________________________**_________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/**mailman/listinfo/freebsd-net >eebsd.org/mailman/listinfo/freebsd-net> To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@**freebsd.org >org> " From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 24 12:12:23 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 590C9924 for ; Tue, 24 Sep 2013 12:12:23 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 141C7281A for ; Tue, 24 Sep 2013 12:12:22 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1VORTY-0002Bn-0i for freebsd-hackers@freebsd.org; Tue, 24 Sep 2013 14:12:20 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 24 Sep 2013 14:12:20 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 24 Sep 2013 14:12:20 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Subject: Re: IO Performance under VMware on LSI RAID controller Date: Tue, 24 Sep 2013 14:12:05 +0200 Lines: 51 Message-ID: References: <207F05D4-9D33-41E8-9258-6C47F266D892@gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6MUKxr58EfAMLngETMfV6CWSFFN3S6DWH" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 In-Reply-To: <207F05D4-9D33-41E8-9258-6C47F266D892@gmail.com> X-Enigmail-Version: 1.5.2 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Sep 2013 12:12:23 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --6MUKxr58EfAMLngETMfV6CWSFFN3S6DWH Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 20/09/2013 15:08, Guy Helmer wrote: > On Sep 19, 2013, at 11:25 AM, Guy Helmer wrote: >=20 >> Normally I build VMware ESXi servers with enterprise-class WD SATA dri= ves and I/O performance in FreeBSD VMs on the servers is fine. >> Whenever I build a VMware ESXi server with a RAID controller, IO perfo= rmance is awful in FreeBSD VMs. I've previously seen this effect with VMw= are ESXi 3ware 9690SA-8I and 9650 RAID controllers, and now I'm seeing si= milar performance with a Dell 6/iR controller. >> >> Any suggestions would be appreciated. >> >> Guy >=20 > (Replying to self due to hint received off-list) >=20 > I seem to remember controllers mentioned previously by FreeBSD device d= river developers that don't deal well with large I/O requests. It turns o= ut that may be the case with VMware device drivers as well -- reducing th= e VMware Disk.DiskMaxIOSize value from its huge default of 32676KB to 32K= B seems to have helped. Disk ops/sec in the FreeBSD VM are now peaking ov= er 400/sec. Interesting that the problem shows only on RAID controllers. Do you have any ideas why this reduction helps (did you find a FAQ or a forum post)? The default RAID stripe size in LSI is 64 KiB, maybe it would help even further to align it also? --6MUKxr58EfAMLngETMfV6CWSFFN3S6DWH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iKYEARECAGYFAlJBgZZfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDYxNDE4MkQ3ODMwNDAwMDJFRUIzNDhFNUZE MDhENTA2M0RGRjFEMkMACgkQ/QjVBj3/HSwqtQCdEs1nccVlzDTOv21yrIkkpTfP s8gAn31S/5nq3DJCr+PusjDrGX0hVsQ/ =RfCu -----END PGP SIGNATURE----- --6MUKxr58EfAMLngETMfV6CWSFFN3S6DWH-- From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 24 12:30:47 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2C1AA392 for ; Tue, 24 Sep 2013 12:30:47 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from ibiza.webweaving.org (ibiza.webweaving.org [204.109.56.32]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 971EF2943 for ; Tue, 24 Sep 2013 12:30:46 +0000 (UTC) Received: from pikmeer.webweaving.org (pikmeer.webweaving.org [178.18.23.51]) by ibiza.webweaving.org (8.14.7/8.14.7) with ESMTP id r8NAfTmg063768; Mon, 23 Sep 2013 10:41:29 GMT (envelope-from dirkx@webweaving.org) Received: from acorn.leiden.webweaving.org (5ED28243.cm-7-3c.dynamic.ziggo.nl [94.210.130.67]) (authenticated bits=0) by pikmeer.webweaving.org (8.14.5/8.14.5) with ESMTP id r8NAfSG6081712 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 23 Sep 2013 10:41:28 GMT (envelope-from dirkx@webweaving.org) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: VPS Project From: Dirk-Willem van Gulik In-Reply-To: <20130923094150.GM12210@mx.7he.at> Date: Mon, 23 Sep 2013 12:41:27 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <381FF194-928A-4483-B011-5A1B0A2EF93B@webweaving.org> References: <20130923094150.GM12210@mx.7he.at> To: "Klaus P. Ohrhallinger" X-Mailer: Apple Mail (2.1510) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (ibiza.webweaving.org [204.109.56.32]); Mon, 23 Sep 2013 10:41:30 +0000 (UTC) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.5 (pikmeer.webweaving.org [178.18.23.51]); Mon, 23 Sep 2013 10:41:28 +0000 (UTC) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Sep 2013 12:30:47 -0000 Op 23 sep. 2013, om 11:41 heeft "Klaus P. Ohrhallinger" het = volgende geschreven: > My virtualization project (http://www.7he.at/freebsd/vps/) has > its project branch on svn.freebsd.org since a few months now. >=20 > Since then there was not much progress due to lack of time. >=20 > Now I am sitting on code updates that I can't commit myself. > What is necessary to get commit right to /projects/vps ? >=20 > Also it would be great if some FreeBSD developers could review > parts of the code. So far, most feedback came from non-developers. Would be nice to see an update (most recent binary packages have drifted = too far to cleany apply); found it very usable for large number of = participating node-testing of network stacks. Dw.= From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 25 20:13:42 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B9785D4E for ; Wed, 25 Sep 2013 20:13:42 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3FA8A20A9 for ; Wed, 25 Sep 2013 20:13:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id r8PKDXPQ041485 for ; Thu, 26 Sep 2013 00:13:33 +0400 (MSK) (envelope-from marck@rinet.ru) Date: Thu, 26 Sep 2013 00:13:33 +0400 (MSK) From: Dmitry Morozovsky To: freebsd-hackers@FreeBSD.org Subject: loader code to zpool.cache loading Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Thu, 26 Sep 2013 00:13:33 +0400 (MSK) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Sep 2013 20:13:42 -0000 Dear collegaues, one of my friends tonight called me to help him with recovering his remote(both to me and him, heh) server with 8.0 or something with ZFS-on root (yes, I'd said he was brave enough) having troubles with loading modules by loader. During investigation it has been clear we need to load /boot/kernel.old/kernel with corresponding modules (including ZFS), but we failed to achieve this, namely: zpool.cache is not loading again, leading to "can't mount root" error. I checked the sources quickly and now I'm a bit puzzled: I see zpool_cache_load="YES" zpool_cache_type="/boot/zfs/zpool.cache" zpool_cache_name="/boot/zfs/zpool.cache" in loader.conf (actually, default one) but can't find references to it anywhere in /sys/boot -- what did I missed? Thanks! -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-hackers@FreeBSD.ORG Fri Sep 27 19:20:40 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C5BC6ED5 for ; Fri, 27 Sep 2013 19:20:40 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm15-vm6.bullet.mail.ne1.yahoo.com (nm15-vm6.bullet.mail.ne1.yahoo.com [98.138.91.108]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 48DB425D9 for ; Fri, 27 Sep 2013 19:20:39 +0000 (UTC) Received: from [98.138.90.50] by nm15.bullet.mail.ne1.yahoo.com with NNFMP; 27 Sep 2013 19:20:33 -0000 Received: from [98.138.226.58] by tm3.bullet.mail.ne1.yahoo.com with NNFMP; 27 Sep 2013 19:20:33 -0000 Received: from [127.0.0.1] by smtp209.mail.ne1.yahoo.com with NNFMP; 27 Sep 2013 19:20:33 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1380309633; bh=F43mny56GiksfN7WUAZqrJIUIenGpoXmpkhdt0Wom6Q=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=BQmC7hvbwovEt8jUgBLIMN+XOBdn1oGzd2KJS+Z0Jaltp+gJaEUiQyj9Uw8mHpbnWYTKMhKPjwhj35FoPbnoEJ0sGsBfkn9+5pis9lk7tBS2BChRmQMfPm5xN57a4uFuko0NRS46+7cED9BJiIONxfhYhZMyR4MzPGFT+DatQsM= X-Yahoo-Newman-Id: 741053.8121.bm@smtp209.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 9LQ1YwYVM1k6uMszsts94.glNcj2WtFPwYDx3Lc25No3rQw TgJJyDBby9wHYkzU5JGw35yV0aVxMtcULLxgJZjtDzI60OU0g1nE.QAqm7Nv QwDf5T0KficM3YVg0Rm0ANBUK4IiXzaEDYAni68cej0WvozMQUy7LKQNZ.YY hsUGEVCvbWtaj4AqlkeF2zLivp4NjdeiO3Ve4YmmSazlp1AmQekblkLa2Bm1 xSV8WqHPOXeiGzp1AxDHcos_Kfh8jip.HNc0R2vavGf83KSUIiUznKlKmlIF VFWrYjoPfeCBY4rfdGWmq44msLyI_40cPw9cmF4GYezLzzdw.N8MgKzSF18w SrVMXmPCw1UAjxgdNvejz31H0JEKszUQqr.THknLLoWS_fSG3.8PL5nHbTtp 5hUjv3TCJnoPFdn3hBezBpakRwAL6Hgd9s9RqM3rS.epMxAnju78xb5VSy3d EecdKvYKJvvKL_qFLdis6fW3XwWmzOW9TITUYdfTnoq0BCKR96U9ksjFshOt LxZoNrVzRUOOCgk6jql_32xJJNipYUlZZDe.CqsJ2QI4PUg1MRzBPBdhitWs JeQ-- X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [10.73.160.242] (sean_bruno@209.131.62.116 with ) by smtp209.mail.ne1.yahoo.com with SMTP; 27 Sep 2013 19:20:33 +0000 UTC Subject: Trying to use /bin/sh From: Sean Bruno To: "freebsd-hackers@freebsd.org" Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-LFrcKzzh3U7+wYnFXmnx" Date: Fri, 27 Sep 2013 12:20:32 -0700 Message-ID: <1380309632.84323.3.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 19:20:40 -0000 --=-LFrcKzzh3U7+wYnFXmnx Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I was attempting to simply set my EDITOR env variable for use on my system today and discovered that I do *not* understand how computers work. After realizing that I simply do not understand what's going on, I broke down and put an "echo test" into /etc/profile and ~/.profile. I can't see why, but logging in via xdm/xfce4 isn't doing any of the normal login things and nothing in my .profile is ever executed by any shell, ever. None of these get executed unless its a login shell. xfce4 doesn't seem to pick these up when starting up, so I have no idea what to do. For now, I've put the needed things into a .bashrc and switched my loging shell to bash, which is not really what I wanted to do. Sean --=-LFrcKzzh3U7+wYnFXmnx Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (FreeBSD) iQEcBAABAgAGBQJSRdp5AAoJEBkJRdwI6BaHfLMH/2zuvDSfNCMsEdIreNvOO8RP R4yfICr3ynjkCqrkCJ0EaGsGyeZkFhlOx9xU9rbGgd+RRV43eU1ZI4B1JXKiRk5h XW1u8IC9rJKtIhj0TvdF+2/JuiUj/+J1H5witlYwppwhJQGgOIDzXqJnfdhaywEd 2vQ4NyU+06b+L/tYa6VNgEkDsXvsJSY6htRC77y3+GaFrDO8rxpZfDUFk6NlkZDg 8qyyCGMijICzDGCn0PtumQQyTR5O3qeSze61y8Pq8666aMHo8ZiHAKDvyayAYUbM Kkh8oaAiDRz7+qYcGqRadihe4mYpTLug6MrOirHQm3jDdy/+Qd7aJL8mN9nWICc= =MYDM -----END PGP SIGNATURE----- --=-LFrcKzzh3U7+wYnFXmnx-- From owner-freebsd-hackers@FreeBSD.ORG Fri Sep 27 22:15:17 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 61C3ABAC; Fri, 27 Sep 2013 22:15:17 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (unknown [IPv6:2001:610:1108:5012::107]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 282E22E35; Fri, 27 Sep 2013 22:15:17 +0000 (UTC) Received: from turtle.stack.nl (turtle.stack.nl [IPv6:2001:610:1108:5010::132]) by mx1.stack.nl (Postfix) with ESMTP id C9A0F1203C2; Sat, 28 Sep 2013 00:14:59 +0200 (CEST) Received: by turtle.stack.nl (Postfix, from userid 1677) id A1702CB4E; Sat, 28 Sep 2013 00:14:59 +0200 (CEST) Date: Sat, 28 Sep 2013 00:14:59 +0200 From: Jilles Tjoelker To: sbruno@freebsd.org Subject: Re: Trying to use /bin/sh Message-ID: <20130927221459.GA27990@stack.nl> References: <1380309632.84323.3.camel@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1380309632.84323.3.camel@localhost> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 22:15:17 -0000 On Fri, Sep 27, 2013 at 12:20:32PM -0700, Sean Bruno wrote: > I was attempting to simply set my EDITOR env variable for use on my > system today and discovered that I do *not* understand how computers > work. > After realizing that I simply do not understand what's going on, I broke > down and put an "echo test" into /etc/profile and ~/.profile. > I can't see why, but logging in via xdm/xfce4 isn't doing any of the > normal login things and nothing in my .profile is ever executed by any > shell, ever. > None of these get executed unless its a login shell. xfce4 doesn't seem > to pick these up when starting up, so I have no idea what to do. > For now, I've put the needed things into a .bashrc and switched my > loging shell to bash, which is not really what I wanted to do. sh's model of startup files (only login shells use startup files with fixed names, other interactive shells only use $ENV) assumes that every session will load /etc/profile and ~/.profile at some point. This includes graphical sessions. The ENV file typically contains only shell options, aliases, function definitions and unexported variables but no environment variables. Some graphical environments actually source shell startup files like ~/.profile when logging in. I remember this from CDE for example. It is important to have some rule where this should happen to avoid doing it twice or never in "strange" configurations. As a workaround, I made ~/.xsession a script interpreted by my login shell and source some startup files. A problem here is that different login shells have incompatible startup files. A different direction is to accept that the environment variables in the X session will be incorrect and make all xterms start login shells (e.g. xterm -ls). You may also be able to use setenv in ~/.login_conf. -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 08:12:37 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0C233E70 for ; Sat, 28 Sep 2013 08:12:37 +0000 (UTC) (envelope-from se@freebsd.org) Received: from nm19-vm0.bullet.mail.ird.yahoo.com (nm19-vm0.bullet.mail.ird.yahoo.com [77.238.189.92]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1112422EE for ; Sat, 28 Sep 2013 08:12:35 +0000 (UTC) Received: from [77.238.189.56] by nm19.bullet.mail.ird.yahoo.com with NNFMP; 28 Sep 2013 08:12:27 -0000 Received: from [46.228.39.71] by tm9.bullet.mail.ird.yahoo.com with NNFMP; 28 Sep 2013 08:12:27 -0000 Received: from [127.0.0.1] by smtp108.mail.ir2.yahoo.com with NNFMP; 28 Sep 2013 08:12:27 -0000 X-Yahoo-Newman-Id: 702683.83249.bm@smtp108.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: mN9iPXwVM1n1_Lug36fVyfvdGjC.Qv6TBXmPvt4DZkg8vzy VX0euHihX3ZM5KeQwUNKJbbFXssBjZUfSKvbdckOKGCNagGMTE5iXCStBkjF HEo5GGFW4f5zXT85FRo5P2yQfGui1BCnX88nrU2KVSC2G7VQR6qAwTW0Ags. ql3viLqzoOeins6Vr5FkL41BHbpaRKvGfDIa.8gK5CeeAQr6Km.RxAxu4VE4 GX3TYbRve9MQgtX.FasSa3rDtg5Zg0wWv7vnubgr2Vj7A7eT2xBjt6VlFL6l ol2QrTF7nJljCHXqqoKTnYMPDxcnUVr48cNYzojIYWUo9OPtd1MU1_2uI45J lw.ZjVoYf0sUKU.80QFLtsFLtL8IB64iaKbhyzHmRFP768y7JKakBC6A9XLj lJijvWO6JHRZYriurnTWkK8D6iz69YW3M6kvMTqHqWBWP3c33fW5vZZ7RGcY jPtClmAoj1I0a15eTxM3fm5EmtUrdj6KAvhx.1d337X27NsNvI_6wO5Kwkxl cmbyxQxK5dgZ6vRYez5BRbCpyeVzgxQ-- X-Yahoo-SMTP: iDf2N9.swBDAhYEh7VHfpgq0lnq. X-Rocket-Received: from [192.168.119.11] (se@84.154.117.89 with ) by smtp108.mail.ir2.yahoo.com with SMTP; 28 Sep 2013 08:12:27 +0000 UTC Message-ID: <52468F62.6010404@freebsd.org> Date: Sat, 28 Sep 2013 10:12:18 +0200 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Trying to use /bin/sh References: <1380309632.84323.3.camel@localhost> <20130927221459.GA27990@stack.nl> In-Reply-To: <20130927221459.GA27990@stack.nl> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 08:12:37 -0000 Am 28.09.2013 00:14, schrieb Jilles Tjoelker: > sh's model of startup files (only login shells use startup files with > fixed names, other interactive shells only use $ENV) assumes that every > session will load /etc/profile and ~/.profile at some point. This > includes graphical sessions. The ENV file typically contains only shell > options, aliases, function definitions and unexported variables but no > environment variables. > > Some graphical environments actually source shell startup files like > ~/.profile when logging in. I remember this from CDE for example. It is > important to have some rule where this should happen to avoid doing it > twice or never in "strange" configurations. As a workaround, I made > ~/.xsession a script interpreted by my login shell and source some > startup files. A problem here is that different login shells have > incompatible startup files. I used to modify Xsession to do the final exec with a forced login shell of the user. This worked for users of all shells. The script identified the shell to use and then used argv0 to start a "login shell" to execute the display manager. A simplified version of my Xsession script is: ------------------------------------------------------------------ #!/bin/sh LIB=/usr/local/lib SH=$SHELL [ -n "$SH" ] || SH="/bin/sh" SHNAME=`basename $SH` echo "exec $LIB/xdm/Xsession.real $*" | \ /usr/local/bin/argv0 $SH -$SHNAME ------------------------------------------------------------------ The argv0 command is part of "sysutils/ucspi-tcp", BTW. This script prepends a "-" to the name of the shell that is started to execute the real "Xsession", which had been renamed to Xession.real. I know that the script could be further simplified by using "modern" variable expansion/substitution commands, but this script was in use some 25 years ago on a variety of Unix systems (SunOS, Ultrix, HP-UX) and I only used the minimal set of Bourne Shell facilities, then. You may want a command to source standard profiles or environment settings before the final exec, in case the users shell does not load them. Regards, STefan From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 10:38:06 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6F094C97 for ; Sat, 28 Sep 2013 10:38:06 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 06E2C2359 for ; Sat, 28 Sep 2013 10:38:05 +0000 (UTC) Received: from server.rulingia.com (c220-239-237-213.belrs5.nsw.optusnet.com.au [220.239.237.213]) by vps.rulingia.com (8.14.7/8.14.5) with ESMTP id r8SAc3cV025181 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Sat, 28 Sep 2013 20:38:04 +1000 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.7/8.14.7) with ESMTP id r8SAbwWn027857 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sat, 28 Sep 2013 20:37:58 +1000 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.7/8.14.7/Submit) id r8SAbwWI027856 for freebsd-hackers@freebsd.org; Sat, 28 Sep 2013 20:37:58 +1000 (EST) (envelope-from peter) Date: Sat, 28 Sep 2013 20:37:58 +1000 From: Peter Jeremy To: freebsd-hackers@freebsd.org Subject: Mixing amd64 kernel with i386 world Message-ID: <20130928103758.GC27231@server.rulingia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="k1lZvvs/B4yU6o8G" Content-Disposition: inline X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 10:38:06 -0000 --k1lZvvs/B4yU6o8G Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I have a system with 4GB RAM and hence need to use an amd64 kernel to use all the RAM (I can only access 3GB RAM with an i386 kernel). OTOH, amd64 processes are significantly (50-100%) larger than equivalent i386 processes and none none of the applications I'll be running on the system need to be 64-bit. This implies that the optimal approach is an amd64 kernel with i386 userland (I'm ignoring PAE as a useable approach). I've successfully run i386 jails on amd64 systems so I know this mostly works. I also know that there are some gotchas: - kdump needs to match the kernel - anything accessing /dev/mem or /dev/kmem (which implies anything that uses libkvm) probably needs to match the kernel. Has anyone investigated this approach? --=20 Peter Jeremy --k1lZvvs/B4yU6o8G Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iEYEARECAAYFAlJGsYYACgkQ/opHv/APuIckqQCgxMZcBfolKIOglm0jy7JeJS+D sAIAoK0+PtR5Bn+uKZmbS2ZeSkZNpTEj =jH8y -----END PGP SIGNATURE----- --k1lZvvs/B4yU6o8G-- From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 11:25:55 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 39B4F850 for ; Sat, 28 Sep 2013 11:25:55 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:7b8:3a7:1:2d0:b7ff:fea0:8c26]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EF2C32625 for ; Sat, 28 Sep 2013 11:25:54 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::c1de:2ac5:7b4a:4aa5] (unknown [IPv6:2001:7b8:3a7:0:c1de:2ac5:7b4a:4aa5]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 7BB375C44; Sat, 28 Sep 2013 13:25:49 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_3EF7EB08-9956-427A-A572-2DD3BAD153E3"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: Mixing amd64 kernel with i386 world From: Dimitry Andric In-Reply-To: <20130928103758.GC27231@server.rulingia.com> Date: Sat, 28 Sep 2013 13:25:39 +0200 Message-Id: <30CAFBF9-4119-4E3D-A5B0-4520E35F1601@FreeBSD.org> References: <20130928103758.GC27231@server.rulingia.com> To: Peter Jeremy X-Mailer: Apple Mail (2.1510) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 11:25:55 -0000 --Apple-Mail=_3EF7EB08-9956-427A-A572-2DD3BAD153E3 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On Sep 28, 2013, at 12:37, Peter Jeremy wrote: > I have a system with 4GB RAM and hence need to use an amd64 kernel to use > all the RAM (I can only access 3GB RAM with an i386 kernel). OTOH, amd64 > processes are significantly (50-100%) larger than equivalent i386 processes > and none none of the applications I'll be running on the system need to be > 64-bit. > > This implies that the optimal approach is an amd64 kernel with i386 > userland (I'm ignoring PAE as a useable approach). I've successfully > run i386 jails on amd64 systems so I know this mostly works. I also > know that there are some gotchas: > - kdump needs to match the kernel > - anything accessing /dev/mem or /dev/kmem (which implies anything that > uses libkvm) probably needs to match the kernel. > > Has anyone investigated this approach? I have only run i386 jails on amd64, just like you. Though for the use case you are describing, the really optimal approach would be to have real x32 support. Unfortunately, that is quite a lot of work... :) -Dimitry --Apple-Mail=_3EF7EB08-9956-427A-A572-2DD3BAD153E3 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.20 (Darwin) iEYEARECAAYFAlJGvLgACgkQsF6jCi4glqPkOQCfc5USkKfbfI38xL9oNLMpy+cN ko0AoLh/IZpYCL2Y85NmLfKihFORz2bM =gXK/ -----END PGP SIGNATURE----- --Apple-Mail=_3EF7EB08-9956-427A-A572-2DD3BAD153E3-- From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 13:56:46 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0A7D5574 for ; Sat, 28 Sep 2013 13:56:46 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D59BE2D81 for ; Sat, 28 Sep 2013 13:56:45 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1VPv0h-000Lmi-By; Sat, 28 Sep 2013 13:56:39 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r8SDuZx8007744; Sat, 28 Sep 2013 07:56:36 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18PMtJ6pl9pbI6a/EJXWyiv Subject: Re: Mixing amd64 kernel with i386 world From: Ian Lepore To: Peter Jeremy In-Reply-To: <20130928103758.GC27231@server.rulingia.com> References: <20130928103758.GC27231@server.rulingia.com> Content-Type: text/plain; charset="us-ascii" Date: Sat, 28 Sep 2013 07:56:35 -0600 Message-ID: <1380376595.1197.309.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 13:56:46 -0000 On Sat, 2013-09-28 at 20:37 +1000, Peter Jeremy wrote: > I have a system with 4GB RAM and hence need to use an amd64 kernel to use > all the RAM (I can only access 3GB RAM with an i386 kernel). OTOH, amd64 > processes are significantly (50-100%) larger than equivalent i386 processes > and none none of the applications I'll be running on the system need to be > 64-bit. > > This implies that the optimal approach is an amd64 kernel with i386 > userland (I'm ignoring PAE as a useable approach). I've successfully > run i386 jails on amd64 systems so I know this mostly works. I also > know that there are some gotchas: > - kdump needs to match the kernel > - anything accessing /dev/mem or /dev/kmem (which implies anything that > uses libkvm) probably needs to match the kernel. > > Has anyone investigated this approach? > Why are you ignoring PAE? It's been working for me for years. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 14:27:29 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6DD6DD39; Sat, 28 Sep 2013 14:27:29 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 05E3A2EDB; Sat, 28 Sep 2013 14:27:28 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r8SERIoj012711; Sat, 28 Sep 2013 17:27:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r8SERIoj012711 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r8SERGuf012710; Sat, 28 Sep 2013 17:27:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 28 Sep 2013 17:27:16 +0300 From: Konstantin Belousov To: Dimitry Andric Subject: Re: Mixing amd64 kernel with i386 world Message-ID: <20130928142716.GN41229@kib.kiev.ua> References: <20130928103758.GC27231@server.rulingia.com> <30CAFBF9-4119-4E3D-A5B0-4520E35F1601@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="CE8aATqwJvGw1INf" Content-Disposition: inline In-Reply-To: <30CAFBF9-4119-4E3D-A5B0-4520E35F1601@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 14:27:29 -0000 --CE8aATqwJvGw1INf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 28, 2013 at 01:25:39PM +0200, Dimitry Andric wrote: > On Sep 28, 2013, at 12:37, Peter Jeremy wrote: > > I have a system with 4GB RAM and hence need to use an amd64 kernel to u= se > > all the RAM (I can only access 3GB RAM with an i386 kernel). OTOH, amd= 64 > > processes are significantly (50-100%) larger than equivalent i386 proce= sses > > and none none of the applications I'll be running on the system need to= be > > 64-bit. > >=20 > > This implies that the optimal approach is an amd64 kernel with i386 > > userland (I'm ignoring PAE as a useable approach). I've successfully > > run i386 jails on amd64 systems so I know this mostly works. I also > > know that there are some gotchas: > > - kdump needs to match the kernel > > - anything accessing /dev/mem or /dev/kmem (which implies anything that > > uses libkvm) probably needs to match the kernel. > >=20 > > Has anyone investigated this approach? >=20 > I have only run i386 jails on amd64, just like you. Though for the use > case you are describing, the really optimal approach would be to have > real x32 support. Unfortunately, that is quite a lot of work... :) The management interfaces usually do not work for compat32 on amd64. The biggest exception I am aware of is nmount(2). But you would be without network. The same is true for almost all sysctls, again kern.proc.* hierarchy is an exception and works. Syscalls required to normal usermode operation do work, at least most developers care about this, except for capsicum. Using 32bit jail or, even easier, chroot, for now, is the easiest propositi= on. --CE8aATqwJvGw1INf Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQIcBAEBAgAGBQJSRudDAAoJEJDCuSvBvK1BOcoP/RlOkjVFE2AYrUzQ1MLb61fi 7zi/xV/Ay+wmuG5M+xxXUUvCrc/EbHM5SayHCOBR87rKlzJah2wgOobfp3/4yDEz dk9wuEbVP9Ic0c1DPLWp3xSCnstiLogwCJb0ggHHLpEJgExb1Xq8b9C1Laga+aEu QTRhFfKQcVxdk2zYN3LNeaoGHfn52Bwjwqemgz69SZkmieUb/EkD+VZxclK7SUwa whyag0fYMoypFFT9/2TJBUcLGspYQ4d5HltyKPrH0MeZEdEDGkY9iNXz2duDC6kn iohxuxg3ixLI/h6NqdPwV1j6DL33M6PdvaqJ81y9JlQQdqKygA9O5kWGgDtucslr cyc/UAXmmp88XtFD3N4Q/jgqF8Qpx7NEHYFIJByhoAwtJ7pKUVRXtTgdqS8myWrz Y60tM8byQZcBsxKOYBtA3VKqW61K+GZnrFycw7ZyIO6WxKWKs6xH+2+HeMYmooGB XtZJLTsAAM+OOm2GqM+7T/UDWJkAAw12R9TEI1Es7VB899ldCijQ9tPG85G7mi0O qaePYUorGU2Pfg1fXrovmap7DechsYejqIhEfM+5O1kCQRW6sSAYNlL8hOWkWp+r pRChyLPpFjFepO87+rHfS1QASEYK4pJvafxzcv7MHolL2d74sLC06JIhWtwAYNei nq4gjNKgq/ZEAx6bb3QV =3fys -----END PGP SIGNATURE----- --CE8aATqwJvGw1INf-- From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 14:34:27 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D2D4BEAF for ; Sat, 28 Sep 2013 14:34:27 +0000 (UTC) (envelope-from nwhitehorn@anacreon.physics.wisc.edu) Received: from anacreon.physics.wisc.edu (unknown [IPv6:2607:f388:101c:0:216:cbff:fe39:3fae]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9CB0F2F3E for ; Sat, 28 Sep 2013 14:34:27 +0000 (UTC) Received: from anacreon.physics.wisc.edu (localhost [127.0.0.1]) by anacreon.physics.wisc.edu (8.14.7/8.14.7) with ESMTP id r8SEYP1U069291; Sat, 28 Sep 2013 09:34:25 -0500 (CDT) (envelope-from nwhitehorn@anacreon.physics.wisc.edu) Received: from localhost (nwhitehorn@localhost) by anacreon.physics.wisc.edu (8.14.7/8.14.7/Submit) with ESMTP id r8SEYPCS069288; Sat, 28 Sep 2013 09:34:25 -0500 (CDT) (envelope-from nwhitehorn@anacreon.physics.wisc.edu) Date: Sat, 28 Sep 2013 09:34:25 -0500 (CDT) From: Nathan Whitehorn To: Peter Jeremy Subject: Re: Mixing amd64 kernel with i386 world In-Reply-To: <20130928103758.GC27231@server.rulingia.com> Message-ID: References: <20130928103758.GC27231@server.rulingia.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Mailman-Approved-At: Sat, 28 Sep 2013 16:09:30 +0000 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 14:34:27 -0000 On Sat, 28 Sep 2013, Peter Jeremy wrote: > I have a system with 4GB RAM and hence need to use an amd64 kernel to use > all the RAM (I can only access 3GB RAM with an i386 kernel). OTOH, amd64 > processes are significantly (50-100%) larger than equivalent i386 processes > and none none of the applications I'll be running on the system need to be > 64-bit. > > This implies that the optimal approach is an amd64 kernel with i386 > userland (I'm ignoring PAE as a useable approach). I've successfully > run i386 jails on amd64 systems so I know this mostly works. I also > know that there are some gotchas: > - kdump needs to match the kernel > - anything accessing /dev/mem or /dev/kmem (which implies anything that > uses libkvm) probably needs to match the kernel. > For whatever it is worth, I have done this running a ppc32 userland with a ppc64 kernel (that is how ppc64 was first developed actually, before the 64-bit userland existed) and it worked just fine. -Nathan From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 16:39:32 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9F79CD09; Sat, 28 Sep 2013 16:39:32 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2244B255E; Sat, 28 Sep 2013 16:39:32 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.16]) by ltcfislmsgpa07.fnfis.com (8.14.5/8.14.5) with ESMTP id r8SGdTZQ020106 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sat, 28 Sep 2013 11:39:31 -0500 Received: from LTCFISWMSGMB21.FNFIS.com ([169.254.1.202]) by LTCFISWMSGHT05.FNFIS.com ([10.132.206.16]) with mapi id 14.02.0309.002; Sat, 28 Sep 2013 11:36:04 -0500 From: "Teske, Devin" To: "sbruno@freebsd.org" Subject: Re: Trying to use /bin/sh Thread-Topic: Trying to use /bin/sh Thread-Index: AQHOvGjSXYG50pxOjUqfcBB22CroTA== Date: Sat, 28 Sep 2013 16:36:03 +0000 Message-ID: <13CA24D6AB415D428143D44749F57D720FC05A80@LTCFISWMSGMB21.FNFIS.com> References: <1380309632.84323.3.camel@localhost> <20130927221459.GA27990@stack.nl> <52468F62.6010404@freebsd.org> In-Reply-To: <52468F62.6010404@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.132.253.121] Content-Type: text/plain; charset="us-ascii" Content-ID: <3CCC4B8D7A6627478CFBBF0EB894C4DC@fisglobal.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794, 1.0.431, 0.0.0000 definitions=2013-09-28_02:2013-09-27,2013-09-28,1970-01-01 signatures=0 Cc: FreeBSD Hackers , Devin Teske X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Devin Teske List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 16:39:32 -0000 On Sep 28, 2013, at 1:12 AM, Stefan Esser wrote: > Am 28.09.2013 00:14, schrieb Jilles Tjoelker: >> sh's model of startup files (only login shells use startup files with >> fixed names, other interactive shells only use $ENV) assumes that every >> session will load /etc/profile and ~/.profile at some point. This >> includes graphical sessions. The ENV file typically contains only shell >> options, aliases, function definitions and unexported variables but no >> environment variables. >>=20 >> Some graphical environments actually source shell startup files like >> ~/.profile when logging in. I remember this from CDE for example. It is >> important to have some rule where this should happen to avoid doing it >> twice or never in "strange" configurations. As a workaround, I made >> ~/.xsession a script interpreted by my login shell and source some >> startup files. A problem here is that different login shells have >> incompatible startup files. >=20 > I used to modify Xsession to do the final exec with a forced login > shell of the user. This worked for users of all shells. >=20 > The script identified the shell to use and then used argv0 to start > a "login shell" to execute the display manager. >=20 > A simplified version of my Xsession script is: >=20 > ------------------------------------------------------------------ > #!/bin/sh >=20 > LIB=3D/usr/local/lib >=20 > SH=3D$SHELL > [ -n "$SH" ] || SH=3D"/bin/sh" > SHNAME=3D`basename $SH` >=20 > echo "exec $LIB/xdm/Xsession.real $*" | \ > /usr/local/bin/argv0 $SH -$SHNAME > ------------------------------------------------------------------ >=20 > The argv0 command is part of "sysutils/ucspi-tcp", BTW. >=20 > This script prepends a "-" to the name of the shell that is > started to execute the real "Xsession", which had been renamed > to Xession.real. >=20 > I know that the script could be further simplified by using "modern" > variable expansion/substitution commands, but this script was in use > some 25 years ago on a variety of Unix systems (SunOS, Ultrix, HP-UX) > and I only used the minimal set of Bourne Shell facilities, then. >=20 > You may want a command to source standard profiles or environment > settings before the final exec, in case the users shell does not > load them. >=20 In my ~/.fvwm2rc file, this is how I launch an XTerm. This achieves the goal of sourcing my profile scripts like a normal login shell while launchi= ng XTerm(s) in the GUI. DestroyFunc FvwmXTerm AddToFunc FvwmXTerm PipeRead '\ cmd=3D"/usr/bin/xterm"; \ [ -x "${cmd}" ] || cmd=3D"/usr/X11R6/bin/xterm"; \ [ -x "${cmd}" ] || cmd=3D"xterm"; \ cmd=3D"${cmd} -sb -sl 400"; \ cmd=3D"${cmd} -ls"; \ cmd=3D"${cmd} -r -si -sk"; \ cmd=3D"${cmd} -fn \\"-misc-fixed-medium-r-*-*-15-*\\""; \ echo "+ I Exec exec ${cmd}"' Essentially producing an XTerm invocation of: xterm -sb -sl 400 -ls -r -si -sk -fn "-misc-fixed-medium-r-*-*-15-*" And everytime I launch an XTerm with that, I get my custom prompt set by ~/.bash_profile. Of course, I'm also a TCSH user, so when I flop over to tcsh, I also get my custom prompt set by ~/.tcshrc But failing that... you could actually make your XTerm a login shell with: xterm -e login But of course, then you're looking at having to enter credentials. Perhaps it's just a matter of getting your commands into the right file... .bash_profile for bash and .tcshrc for tcsh. --=20 Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you. From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 16:49:16 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 95D973A8; Sat, 28 Sep 2013 16:49:16 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 57B2F260B; Sat, 28 Sep 2013 16:49:16 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1VPxhd-000PAN-Fl; Sat, 28 Sep 2013 16:49:09 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r8SGn5nP007861; Sat, 28 Sep 2013 10:49:05 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX19Kzbuco+n+PFyfLNRjdbL5 Subject: Re: Trying to use /bin/sh From: Ian Lepore To: Devin Teske In-Reply-To: <13CA24D6AB415D428143D44749F57D720FC05A80@LTCFISWMSGMB21.FNFIS.com> References: <1380309632.84323.3.camel@localhost> <20130927221459.GA27990@stack.nl> <52468F62.6010404@freebsd.org> <13CA24D6AB415D428143D44749F57D720FC05A80@LTCFISWMSGMB21.FNFIS.com> Content-Type: text/plain; charset="us-ascii" Date: Sat, 28 Sep 2013 10:49:05 -0600 Message-ID: <1380386945.1197.327.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 16:49:16 -0000 On Sat, 2013-09-28 at 16:36 +0000, Teske, Devin wrote: > On Sep 28, 2013, at 1:12 AM, Stefan Esser wrote: > > > Am 28.09.2013 00:14, schrieb Jilles Tjoelker: > >> sh's model of startup files (only login shells use startup files with > >> fixed names, other interactive shells only use $ENV) assumes that every > >> session will load /etc/profile and ~/.profile at some point. This > >> includes graphical sessions. The ENV file typically contains only shell > >> options, aliases, function definitions and unexported variables but no > >> environment variables. > >> > >> Some graphical environments actually source shell startup files like > >> ~/.profile when logging in. I remember this from CDE for example. It is > >> important to have some rule where this should happen to avoid doing it > >> twice or never in "strange" configurations. As a workaround, I made > >> ~/.xsession a script interpreted by my login shell and source some > >> startup files. A problem here is that different login shells have > >> incompatible startup files. > > > > I used to modify Xsession to do the final exec with a forced login > > shell of the user. This worked for users of all shells. > > > > The script identified the shell to use and then used argv0 to start > > a "login shell" to execute the display manager. > > > > A simplified version of my Xsession script is: > > > > ------------------------------------------------------------------ > > #!/bin/sh > > > > LIB=/usr/local/lib > > > > SH=$SHELL > > [ -n "$SH" ] || SH="/bin/sh" > > SHNAME=`basename $SH` > > > > echo "exec $LIB/xdm/Xsession.real $*" | \ > > /usr/local/bin/argv0 $SH -$SHNAME > > ------------------------------------------------------------------ > > > > The argv0 command is part of "sysutils/ucspi-tcp", BTW. > > > > This script prepends a "-" to the name of the shell that is > > started to execute the real "Xsession", which had been renamed > > to Xession.real. > > > > I know that the script could be further simplified by using "modern" > > variable expansion/substitution commands, but this script was in use > > some 25 years ago on a variety of Unix systems (SunOS, Ultrix, HP-UX) > > and I only used the minimal set of Bourne Shell facilities, then. > > > > You may want a command to source standard profiles or environment > > settings before the final exec, in case the users shell does not > > load them. > > > > In my ~/.fvwm2rc file, this is how I launch an XTerm. This achieves the > goal of sourcing my profile scripts like a normal login shell while launching > XTerm(s) in the GUI. > > DestroyFunc FvwmXTerm > AddToFunc FvwmXTerm > PipeRead '\ > cmd="/usr/bin/xterm"; \ > [ -x "${cmd}" ] || cmd="/usr/X11R6/bin/xterm"; \ > [ -x "${cmd}" ] || cmd="xterm"; \ > cmd="${cmd} -sb -sl 400"; \ > cmd="${cmd} -ls"; \ > cmd="${cmd} -r -si -sk"; \ > cmd="${cmd} -fn \\"-misc-fixed-medium-r-*-*-15-*\\""; \ > echo "+ I Exec exec ${cmd}"' > > Essentially producing an XTerm invocation of: > > xterm -sb -sl 400 -ls -r -si -sk -fn "-misc-fixed-medium-r-*-*-15-*" > > And everytime I launch an XTerm with that, I get my custom prompt set > by ~/.bash_profile. > > Of course, I'm also a TCSH user, so when I flop over to tcsh, I also get > my custom prompt set by ~/.tcshrc > > But failing that... you could actually make your XTerm a login shell with: > > xterm -e login > > But of course, then you're looking at having to enter credentials. > > Perhaps it's just a matter of getting your commands into the right file... > > .bash_profile for bash and .tcshrc for tcsh. For bash the solution I've been using for like 15 years is that my .bash_profile (used only for a login) contains simply: if [ -f ~/.bashrc ]; then . ~/.bashrc fi And everything goes into .bashrc which runs on non-login shell invocation. I have a few lines of code in .bashrc that have to cope with things like not blindly adding something to PATH that's already there[1] but other than that I generally want all the same things to happen whether its a login shell or not. I think the bourne-shell equivelent is to have a .profile that just sets ENV=~/.shrc or similar. (I think someone mentioned that earlier in the thread.) [1] for example: if [[ "$PATH" != *$HOME/bin* && -d $HOME/bin ]] ; then export PATH=$HOME/bin:$PATH fi -- Ian From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 18:32:59 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E37A65C8; Sat, 28 Sep 2013 18:32:59 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9914B2BA5; Sat, 28 Sep 2013 18:32:59 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.16]) by ltcfislmsgpa04.fnfis.com (8.14.5/8.14.5) with ESMTP id r8SIWmq3031807 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sat, 28 Sep 2013 13:32:58 -0500 Received: from LTCFISWMSGMB21.FNFIS.com ([169.254.1.202]) by LTCFISWMSGHT05.FNFIS.com ([10.132.206.16]) with mapi id 14.02.0309.002; Sat, 28 Sep 2013 13:31:49 -0500 From: "Teske, Devin" To: FreeBSD Hackers Subject: bash_profile, tcshrc, etc... Oh My! Thread-Topic: bash_profile, tcshrc, etc... Oh My! Thread-Index: AQHOvHj9t49mWPxWj0mGPAz9D0i1TA== Date: Sat, 28 Sep 2013 18:31:48 +0000 Message-ID: <13CA24D6AB415D428143D44749F57D720FC060FF@LTCFISWMSGMB21.FNFIS.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.132.253.121] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794, 1.0.431, 0.0.0000 definitions=2013-09-28_02:2013-09-27,2013-09-28,1970-01-01 signatures=0 Cc: Devin Teske X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Devin Teske List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 18:33:00 -0000 "Trying to use /bin/sh" thread inspired me to check my home-dir goodies int= o CVS: http://druidbsd.cvs.sf.net/viewvc/druidbsd/home/ Just thought I'd share. There's some cool stuff if you want to have some fu= n. --=20 Cheers, Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you. From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 19:48:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DACDABB1; Sat, 28 Sep 2013 19:48:48 +0000 (UTC) (envelope-from fullermd@over-yonder.net) Received: from thyme.infocus-llc.com (server.infocus-llc.com [206.156.254.44]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ADF942EE9; Sat, 28 Sep 2013 19:48:48 +0000 (UTC) Received: from draco.over-yonder.net (c-75-65-60-66.hsd1.ms.comcast.net [75.65.60.66]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by thyme.infocus-llc.com (Postfix) with ESMTPSA id EED3A37B4AC; Sat, 28 Sep 2013 14:41:47 -0500 (CDT) Received: by draco.over-yonder.net (Postfix, from userid 100) id 3cnKy32P8czVPL; Sat, 28 Sep 2013 14:41:47 -0500 (CDT) Date: Sat, 28 Sep 2013 14:41:47 -0500 From: "Matthew D. Fuller" To: Devin Teske Subject: Re: Trying to use /bin/sh Message-ID: <20130928194147.GG15314@over-yonder.net> References: <1380309632.84323.3.camel@localhost> <20130927221459.GA27990@stack.nl> <52468F62.6010404@freebsd.org> <13CA24D6AB415D428143D44749F57D720FC05A80@LTCFISWMSGMB21.FNFIS.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <13CA24D6AB415D428143D44749F57D720FC05A80@LTCFISWMSGMB21.FNFIS.com> X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.21-fullermd.4 (2010-09-15) X-Virus-Scanned: clamav-milter 0.97.8 at thyme.infocus-llc.com X-Virus-Status: Clean Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 19:48:48 -0000 On Sat, Sep 28, 2013 at 04:36:03PM +0000 I heard the voice of Teske, Devin, and lo! it spake thus: > > xterm -sb -sl 400 -ls -r -si -sk -fn "-misc-fixed-medium-r-*-*-15-*" ^^^ being the operative bit. Or setting loginShell in resources; that way you don't have to change any invocations anywhere. -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 19:58:04 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 31166D9E; Sat, 28 Sep 2013 19:58:04 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F03282F61; Sat, 28 Sep 2013 19:58:03 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.15]) by ltcfislmsgpa07.fnfis.com (8.14.5/8.14.5) with ESMTP id r8SJw2Dg023098 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sat, 28 Sep 2013 14:58:02 -0500 Received: from LTCFISWMSGMB21.FNFIS.com ([169.254.1.202]) by LTCFISWMSGHT04.FNFIS.com ([10.132.206.15]) with mapi id 14.02.0309.002; Sat, 28 Sep 2013 14:58:01 -0500 From: "Teske, Devin" To: "Matthew D. Fuller" Subject: Re: Trying to use /bin/sh Thread-Topic: Trying to use /bin/sh Thread-Index: AQHOvGjSXYG50pxOjUqfcBB22CroTA== Date: Sat, 28 Sep 2013 19:58:00 +0000 Message-ID: <13CA24D6AB415D428143D44749F57D720FC066C2@LTCFISWMSGMB21.FNFIS.com> References: <1380309632.84323.3.camel@localhost> <20130927221459.GA27990@stack.nl> <52468F62.6010404@freebsd.org> <13CA24D6AB415D428143D44749F57D720FC05A80@LTCFISWMSGMB21.FNFIS.com> <20130928194147.GG15314@over-yonder.net> In-Reply-To: <20130928194147.GG15314@over-yonder.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.132.253.121] Content-Type: text/plain; charset="us-ascii" Content-ID: <0856CCC4C23B294CB8516E3F1CE9B950@fisglobal.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794, 1.0.431, 0.0.0000 definitions=2013-09-28_02:2013-09-27,2013-09-28,1970-01-01 signatures=0 Cc: FreeBSD Hackers , Devin Teske X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Devin Teske List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 19:58:04 -0000 On Sep 28, 2013, at 12:41 PM, Matthew D. Fuller wrote: > On Sat, Sep 28, 2013 at 04:36:03PM +0000 I heard the voice of > Teske, Devin, and lo! it spake thus: >>=20 >> xterm -sb -sl 400 -ls -r -si -sk -fn "-misc-fixed-medium-r-*-*-15-*" > ^^^ >=20 For those with variable-width fonts in their Mail Reader... He highlighted "-ls" which stands for "login shell"; xterm(1). > being the operative bit. Or setting loginShell in resources; that way > you don't have to change any invocations anywhere. >=20 Hadn't realized you could do that... excellent! Thanks for the tip on setting that as a resource. --=20 Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you. From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 21:30:29 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DF7B4E7E; Sat, 28 Sep 2013 21:30:29 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ADC9123BF; Sat, 28 Sep 2013 21:30:29 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.15]) by ltcfislmsgpa05.fnfis.com (8.14.5/8.14.5) with ESMTP id r8SLUSBT026420 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sat, 28 Sep 2013 16:30:28 -0500 Received: from LTCFISWMSGMB21.FNFIS.com ([169.254.1.202]) by LTCFISWMSGHT04.FNFIS.com ([10.132.206.15]) with mapi id 14.02.0309.002; Sat, 28 Sep 2013 16:30:27 -0500 From: "Teske, Devin" To: FreeBSD Hackers Subject: ZFS Boot Menu Thread-Topic: ZFS Boot Menu Thread-Index: AQHOvJHy7hpD2Osvt02Q7ZHQzImI/A== Date: Sat, 28 Sep 2013 21:30:26 +0000 Message-ID: <13CA24D6AB415D428143D44749F57D720FC06B52@LTCFISWMSGMB21.FNFIS.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.132.253.121] Content-Type: text/plain; charset="us-ascii" Content-ID: <46B74335C0497D4F9B55208F1BCF6DD2@fisglobal.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794, 1.0.431, 0.0.0000 definitions=2013-09-28_02:2013-09-27,2013-09-28,1970-01-01 signatures=0 Cc: Devin Teske X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Devin Teske List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 21:30:29 -0000 In my recent interview on bsdnow.tv, I was pinged on BEs in Forth. I'd like to revisit this. Back on Sept 20th, 2012, I posted some pics demonstrating what exactly code that was in HEAD (at the time) was/is capable of. These three pictures (posted the same day) tell a story: 1. You boot to the menu: http://twitpic.com/b1eswi/full 2. You select option #5 to get here: http://twitpic.com/b1etyb/full 3. You select option #2 to get here: http://twitpic.com/b1ew47/full I've just (today) uploaded the /boot/menu.rc file(s) that I used to create those pictures: http://druidbsd.cvs.sf.net/viewvc/druidbsd/zfsbeastie/ NB: There's a README file to go along with the files. HINT: diff -pu menu.rc.1.current-head menu.rc.2.cycler HINT: diff -pu menu.rc.1.current-head menu.rc.2.dynamic-submenu Interested in feedback, but moreover I would like to see who is interested in tackling this with me? I can't do it alone... I at least need testers whom will provide feedback and edge-case testing. --=20 Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you. From owner-freebsd-hackers@FreeBSD.ORG Sat Sep 28 23:49:19 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CC43F72F; Sat, 28 Sep 2013 23:49:19 +0000 (UTC) (envelope-from oliver.pntr@gmail.com) Received: from mail-ob0-x231.google.com (mail-ob0-x231.google.com [IPv6:2607:f8b0:4003:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8B8E82A2E; Sat, 28 Sep 2013 23:49:19 +0000 (UTC) Received: by mail-ob0-f177.google.com with SMTP id wp18so4176603obc.36 for ; Sat, 28 Sep 2013 16:49:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Hbbgd9P2CKKLZz5yWIGZoXfz1gyZEMJ+v32JYp3bDv4=; b=Y2oFxEWCeIVB65RqEl85FLtRmDMJ6upJD2w2ovVd2afYVnMf+ZhZMMElh94cxpY6Nn 1smF34YbZznHXApsCOQhHtUdCHDkzgwnan11kaODVNOH0toYvdQXqcLa6tk726S/wssc y0caDJsOjVqa3dIGytkWVkSrnSqPBRqNB70hntnaoFdz7JM4wCyjoNFV5mq7jMabxYiO JfsC6+O6fb1bm1mgFiZCtPfp2PY4Mv/lvJtp+qjSuBBjIgXO8A7SKt8T7eF1rMgahLNK SOg/Q+bvnX7Qp4LGoz+KD2YKQ2PbEI3U5qy++M5Ba51dgz/OpxLMSkk0AXF/NMR33IKI 0meQ== MIME-Version: 1.0 X-Received: by 10.182.99.231 with SMTP id et7mr12773707obb.10.1380412158798; Sat, 28 Sep 2013 16:49:18 -0700 (PDT) Received: by 10.182.22.161 with HTTP; Sat, 28 Sep 2013 16:49:18 -0700 (PDT) In-Reply-To: <13CA24D6AB415D428143D44749F57D720FC060FF@LTCFISWMSGMB21.FNFIS.com> References: <13CA24D6AB415D428143D44749F57D720FC060FF@LTCFISWMSGMB21.FNFIS.com> Date: Sun, 29 Sep 2013 01:49:18 +0200 Message-ID: Subject: Re: bash_profile, tcshrc, etc... Oh My! From: Oliver Pinter To: Devin Teske Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 23:49:19 -0000 On 9/28/13, Teske, Devin wrote: > "Trying to use /bin/sh" thread inspired me to check my home-dir goodies into > CVS: > > http://druidbsd.cvs.sf.net/viewvc/druidbsd/home/ > > Just thought I'd share. There's some cool stuff if you want to have some > fun. > -- > Cheers, > Devin FYI: https://github.com/opntr/freebsd-user-env > > _____________ > The information contained in this message is proprietary and/or > confidential. If you are not the intended recipient, please: (i) delete the > message and all copies; (ii) do not disclose, distribute or use the message > in any manner; and (iii) notify the sender immediately. In addition, please > be aware that any message addressed to our domain is subject to archiving > and review by persons other than the intended recipient. Thank you. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >