From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 3 15:45:37 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C09D91A1; Sun, 3 Mar 2013 15:45:37 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id 84E7FCF2; Sun, 3 Mar 2013 15:45:37 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a] (spaceball.andric.com [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 0AD4B5C43; Sun, 3 Mar 2013 16:45:29 +0100 (CET) Message-ID: <51337018.30000@FreeBSD.org> Date: Sun, 03 Mar 2013 16:45:28 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20130117 Thunderbird/19.0 MIME-Version: 1.0 To: Andriy Gapon , freebsd-hackers@FreeBSD.org Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <51323C62.4040506@FreeBSD.org> <51325FB3.7080300@FreeBSD.org> In-Reply-To: <51325FB3.7080300@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Mar 2013 15:45:37 -0000 On 2013-03-02 21:23, Dimitry Andric wrote: > On 2013-03-02 18:52, Andriy Gapon wrote: >> on 02/03/2013 19:35 Andriy Gapon said the following: >>> Now, I am not quite sure why ctfconvert skips bpobj_iterate_impl in the >>> clang-generated code. Seems like some sort of a bug in ctfconvert. >> >> It seems that gcc and clang put different names for symbol of type FILE: >> clang: >> readelf -a -W /usr/obj/usr/src/sys/TRANT/bpobj.o| fgrep -w FILE >> 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/bpobj.c >> >> gcc: >> readelf -a -W /usr/obj/usr/src/sys/ODYSSEY/bpobj.o| fgrep -w FILE >> 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS bpobj.c >> >> ctfconvert seems to compare this value with "bpobj.c" and so in the clang case >> it doesn't recognize the static symbols. >> >> Does my analysis seem reasonable? > > Have you verified that ctfconvert does the right thing, if you modify > the FILE symbol to have just the filename? Debug log of ctfconvert operating on a gcc-compiled bpobj.o: http://www.andric.com/freebsd/ctfconvert-bpobj-gcc.log The same, but on a clang-compiled bpobj.o: http://www.andric.com/freebsd/ctfconvert-bpobj-clang.log This latter log does not change at all, if you change the FILE symbol to just "bpobj.c". So there seems to something else in (probably) the debug information that confuses ctfconvert. From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 3 16:33:51 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D5AAE92; Sun, 3 Mar 2013 16:33:51 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C86AEEB5; Sun, 3 Mar 2013 16:33:50 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA28767; Sun, 03 Mar 2013 18:33:48 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UCBr9-000Fh6-Tk; Sun, 03 Mar 2013 18:33:48 +0200 Message-ID: <51337B69.8040507@FreeBSD.org> Date: Sun, 03 Mar 2013 18:33:45 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: Dimitry Andric Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <51323C62.4040506@FreeBSD.org> <51325FB3.7080300@FreeBSD.org> In-Reply-To: <51325FB3.7080300@FreeBSD.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Mar 2013 16:33:51 -0000 on 02/03/2013 22:23 Dimitry Andric said the following: > > Have you verified that ctfconvert does the right thing, if you modify > the FILE symbol to have just the filename? No, I haven't. How can I test that? However my reading of the code makes me believe that that would help. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 3 16:36:58 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D0F7519A; Sun, 3 Mar 2013 16:36:58 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D5DB1EC7; Sun, 3 Mar 2013 16:36:57 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA28807; Sun, 03 Mar 2013 18:36:56 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UCBuC-000FhV-6t; Sun, 03 Mar 2013 18:36:56 +0200 Message-ID: <51337C27.7000105@FreeBSD.org> Date: Sun, 03 Mar 2013 18:36:55 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: Dimitry Andric Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <51323C62.4040506@FreeBSD.org> <51325FB3.7080300@FreeBSD.org> <51337018.30000@FreeBSD.org> In-Reply-To: <51337018.30000@FreeBSD.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Mar 2013 16:36:58 -0000 on 03/03/2013 17:45 Dimitry Andric said the following: > Debug log of ctfconvert operating on a gcc-compiled bpobj.o: > > http://www.andric.com/freebsd/ctfconvert-bpobj-gcc.log > > The same, but on a clang-compiled bpobj.o: > > http://www.andric.com/freebsd/ctfconvert-bpobj-clang.log > > This latter log does not change at all, if you change the FILE symbol to > just "bpobj.c". So there seems to something else in (probably) the > debug information that confuses ctfconvert. Sorry, but these two logs look so different to each other and your clang log is so different from a log that I get here with that I believe that something is wrong in your test. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 3 20:02:07 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 208CAD4F; Sun, 3 Mar 2013 20:02:07 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id 748266F7; Sun, 3 Mar 2013 20:02:06 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a] (spaceball.andric.com [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 2E3F55C43; Sun, 3 Mar 2013 21:02:05 +0100 (CET) Message-ID: <5133AC3B.5040504@FreeBSD.org> Date: Sun, 03 Mar 2013 21:02:03 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20130117 Thunderbird/19.0 MIME-Version: 1.0 To: Andriy Gapon Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <51323C62.4040506@FreeBSD.org> <51325FB3.7080300@FreeBSD.org> <51337018.30000@FreeBSD.org> <51337C27.7000105@FreeBSD.org> In-Reply-To: <51337C27.7000105@FreeBSD.org> Content-Type: multipart/mixed; boundary="------------090508090006040708000301" Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Mar 2013 20:02:07 -0000 This is a multi-part message in MIME format. --------------090508090006040708000301 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 2013-03-03 17:36, Andriy Gapon wrote: > on 03/03/2013 17:45 Dimitry Andric said the following: >> Debug log of ctfconvert operating on a gcc-compiled bpobj.o: >> >> http://www.andric.com/freebsd/ctfconvert-bpobj-gcc.log >> >> The same, but on a clang-compiled bpobj.o: >> >> http://www.andric.com/freebsd/ctfconvert-bpobj-clang.log >> >> This latter log does not change at all, if you change the FILE symbol to >> just "bpobj.c". So there seems to something else in (probably) the >> debug information that confuses ctfconvert. > > Sorry, but these two logs look so different to each other and your clang log is > so different from a log that I get here with that I believe that something is > wrong in your test. Indeed, apparently I copy/pasted the wrong command line from earlier in the build log, where it does not use -g (it is probably for the userland version of ctf). When I used the command line for the kernel build of bpobj.c, it did have -g, and the ctfconvert log had a lot more information. I have re-uploaded the logfile on the second URL above. I also had a look at ctfconvert, and it appears that in its dwarf reader, in cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c, it uses the basename, see lines 1825 through 1827: 1762 int 1763 dw_read(tdata_t *td, Elf *elf, char *filename __unused) 1764 { [...] 1824 if ((dw.dw_cuname = die_name(&dw, cu)) != NULL) { 1825 char *base = xstrdup(basename(dw.dw_cuname)); 1826 free(dw.dw_cuname); 1827 dw.dw_cuname = base; 1828 1829 debug(1, "CU name: %s\n", dw.dw_cuname); 1830 } while later on, in actually outputting the processed symbol table, in cddl/contrib/opensolaris/tools/ctf/cvt/output.c, it does not take the basename, but the full name: 336 static iiburst_t * 337 sort_iidescs(Elf *elf, const char *file, tdata_t *td, int fuzzymatch, 338 int dynsym) 339 { [...] 364 for (i = 0; i < nent; i++) { 365 GElf_Sym sym; [...] 372 if (gelf_getsym(data, i, &sym) == NULL) 373 elfterminate(file, "Couldn't read symbol %d", i); 374 375 match.iim_name = (char *)strdata->d_buf + sym.st_name; 376 match.iim_bind = GELF_ST_BIND(sym.st_info); 377 378 switch (GELF_ST_TYPE(sym.st_info)) { 379 case STT_FILE: 380 match.iim_file = match.iim_name; 381 continue; [...] 397 iidesc = find_iidesc(td, &match); 398 399 if (iidesc != NULL) { 400 tolist[*curr] = iidesc; 401 iidesc->ii_flags |= IIDESC_F_USED; 402 (*curr)++; 403 continue; 404 } On line 375, the full name is gotten from the elf symbol, on line 379 it is assigned to the iidesc_match_t struct which is used for searching, and in line 397 it does the actual search. So to fix this inconsistency, we could either change dw_read() to use the full name instead of the basename, or change sort_iidescs() to use the basename instead of the full name. The only upstream I have been able to find, Illumos, had the same code as FreeBSD for both of these functions, so it is likely they never noticed it before... My preference would be to remove the basename() call from dw_read(), as in the attached diff. That seems to fix the problem here; the diff of ctfdump output before and after becomes: --- ctfdump-before.log 2013-03-03 20:37:30.000000000 +0100 +++ ctfdump-after.log 2013-03-03 20:37:47.000000000 +0100 @@ -9,8 +9,8 @@ cth_lbloff = 0 cth_objtoff = 8 cth_funcoff = 10 - cth_typeoff = 140 - cth_stroff = 24164 + cth_typeoff = 160 + cth_stroff = 24184 cth_strlen = 20066 - Label Table ---------------------------------------------------------------- @@ -23,6 +23,8 @@ - Functions ------------------------------------------------------------------ + [0] FUNC (bpobj_iterate_impl) returns: 3 args: (1089, 1136, 56, 1092, 1163) + [1] FUNC (space_range_cb) returns: 3 args: (56, 1124, 1092) [2] FUNC (bpobj_alloc) returns: 324 args: (1080, 1092, 3) [3] FUNC (bpobj_alloc_empty) returns: 324 args: (1080, 3, 1092) [4] FUNC (bpobj_close) returns: 55 args: (1089) @@ -5099,10 +5101,10 @@ total number of data objects = 1 - total number of functions = 12 - total number of function arguments = 39 + total number of functions = 14 + total number of function arguments = 47 maximum argument list length = 6 - average argument list length = 3.25 + average argument list length = 3.36 total number of types = 1173 total number of integers = 18 --------------090508090006040708000301 Content-Type: text/x-diff; name="ctfconvert-dwarf-1.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ctfconvert-dwarf-1.diff" Index: cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c =================================================================== --- cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c (revision 247448) +++ cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c (working copy) @@ -1822,10 +1822,6 @@ dw_read(tdata_t *td, Elf *elf, char *filename __un } if ((dw.dw_cuname = die_name(&dw, cu)) != NULL) { - char *base = xstrdup(basename(dw.dw_cuname)); - free(dw.dw_cuname); - dw.dw_cuname = base; - debug(1, "CU name: %s\n", dw.dw_cuname); } --------------090508090006040708000301-- From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 4 07:57:42 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 97AF2E92 for ; Mon, 4 Mar 2013 07:57:42 +0000 (UTC) (envelope-from mattblists@icritical.com) Received: from mail3.icritical.com (mail3.icritical.com [212.57.248.143]) by mx1.freebsd.org (Postfix) with SMTP id 081BB241 for ; Mon, 4 Mar 2013 07:57:41 +0000 (UTC) Received: (qmail 27220 invoked from network); 4 Mar 2013 07:57:37 -0000 Received: from localhost (127.0.0.1) by mail3.icritical.com with SMTP; 4 Mar 2013 07:57:37 -0000 Received: (qmail 27204 invoked by uid 599); 4 Mar 2013 07:57:30 -0000 Received: from unknown (HELO PDC002.icritical.int) (212.57.254.146) by mail3.icritical.com (qpsmtpd/0.28) with ESMTP; Mon, 04 Mar 2013 07:57:30 +0000 Message-ID: <513453D4.4010305@icritical.com> Date: Mon, 4 Mar 2013 07:57:08 +0000 From: Matt Burke User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130122 Thunderbird/17.0.2 MIME-Version: 1.0 To: Andriy Gapon Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> In-Reply-To: <5132387E.8010808@FreeBSD.org> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-TLS-Incoming: YES X-Virus-Scanned: by iCritical at mail3.icritical.com Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 07:57:42 -0000 On 03/02/13 17:35, Andriy Gapon wrote: > To summarize: I would be glad of either clang generated code was > "fbt-friendly" or if ctf information was generated for > bpobj_iterate_impl. Either is perfect for me. Apologies if this is a silly suggestion, but are you building with -O0? I've noticed backtraces being hard to follow (i.e. skipping intermediate and wrapper functions) with a kernel built with the default optimisation level, and it seems to me that the same thing may be happening here - the wrappers are being optimised out. -- Sorry for the following iCritical is a brand of Critical Software Ltd. Registered in England & Wales: 04909220. Registered Office: IC2, Keele Science Park, Keele, Staffordshire, ST5 5NH. This message has been scanned for security threats by iCritical. The information contained in this message is confidential and intended for the addressee only. If you have received this message in error, or there are any problems with its content, please contact the sender. From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 4 09:16:07 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E3F72238; Mon, 4 Mar 2013 09:16:07 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id 8D3FE81E; Mon, 4 Mar 2013 09:16:07 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a] (spaceball.andric.com [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 29CC35C43; Mon, 4 Mar 2013 10:16:04 +0100 (CET) Message-ID: <51346650.90200@FreeBSD.org> Date: Mon, 04 Mar 2013 10:16:00 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20130117 Thunderbird/19.0 MIME-Version: 1.0 To: Matt Burke , Andriy Gapon Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <513453D4.4010305@icritical.com> In-Reply-To: <513453D4.4010305@icritical.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 09:16:08 -0000 On 2013-03-04 08:57, Matt Burke wrote: > On 03/02/13 17:35, Andriy Gapon wrote: >> To summarize: I would be glad of either clang generated code was >> "fbt-friendly" or if ctf information was generated for >> bpobj_iterate_impl. Either is perfect for me. > > Apologies if this is a silly suggestion, but are you building with -O0? > I've noticed backtraces being hard to follow (i.e. skipping intermediate > and wrapper functions) with a kernel built with the default optimisation > level, and it seems to me that the same thing may be happening here - the > wrappers are being optimised out. No, the wrappers (bpobj_iterate and bpobj_iterate_nofree) are externally visible functions, so they cannot be optimized out. Both of them call bpobj_iterate_impl, which is a static function, e.g. a candidate for automatic inlining. However, bpobj_iterate_impl is quite big, and called from 3 different locations, so it does not actually get inlined, even with -O2. From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 4 11:15:35 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1B471C39 for ; Mon, 4 Mar 2013 11:15:35 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 74804106B for ; Mon, 4 Mar 2013 11:15:34 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA07539; Mon, 04 Mar 2013 13:15:23 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <5134824B.3000301@FreeBSD.org> Date: Mon, 04 Mar 2013 13:15:23 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: Matt Burke Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <513453D4.4010305@icritical.com> In-Reply-To: <513453D4.4010305@icritical.com> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 11:15:35 -0000 on 04/03/2013 09:57 Matt Burke said the following: > On 03/02/13 17:35, Andriy Gapon wrote: > >> To summarize: I would be glad of either clang generated code was >> "fbt-friendly" or if ctf information was generated for >> bpobj_iterate_impl. Either is perfect for me. > > Apologies if this is a silly suggestion, Apologies, but yes, it is :-) > but are you building with -O0? > I've noticed backtraces being hard to follow (i.e. skipping intermediate > and wrapper functions) with a kernel built with the default optimisation > level, and it seems to me that the same thing may be happening here - the > wrappers are being optimised out. > If they were optimized out, then I wouldn't be able to provide their disassembly in my original email, would I? -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 4 11:50:06 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2F2A4D51; Mon, 4 Mar 2013 11:50:06 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id E4BA8125A; Mon, 4 Mar 2013 11:50:05 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a] (spaceball.andric.com [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 929855C43; Mon, 4 Mar 2013 12:50:04 +0100 (CET) Message-ID: <51348A6A.3070606@FreeBSD.org> Date: Mon, 04 Mar 2013 12:50:02 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20130117 Thunderbird/19.0 MIME-Version: 1.0 To: Andriy Gapon Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <51323C62.4040506@FreeBSD.org> <51325FB3.7080300@FreeBSD.org> <51337018.30000@FreeBSD.org> <51337C27.7000105@FreeBSD.org> <5133AC3B.5040504@FreeBSD.org> In-Reply-To: <5133AC3B.5040504@FreeBSD.org> Content-Type: multipart/mixed; boundary="------------060403060209040404050200" Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 11:50:06 -0000 This is a multi-part message in MIME format. --------------060403060209040404050200 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 2013-03-03 21:02, Dimitry Andric wrote: > On 2013-03-03 17:36, Andriy Gapon wrote: >> on 03/03/2013 17:45 Dimitry Andric said the following: ... > So to fix this inconsistency, we could either change dw_read() to use > the full name instead of the basename, or change sort_iidescs() to use > the basename instead of the full name. > > The only upstream I have been able to find, Illumos, had the same code > as FreeBSD for both of these functions, so it is likely they never > noticed it before... > > My preference would be to remove the basename() call from dw_read(), as > in the attached diff. That seems to fix the problem here; the diff of > ctfdump output before and after becomes: Actually, that way of fixing breaks it for gcc, so it should really be fixed in sort_iidescs() instead. See attached diff; I verified it works for both gcc-compiled and clang-compiled objects. --------------060403060209040404050200 Content-Type: text/x-diff; name="ctfconvert-output-1.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ctfconvert-output-1.diff" Index: cddl/contrib/opensolaris/tools/ctf/cvt/output.c =================================================================== --- cddl/contrib/opensolaris/tools/ctf/cvt/output.c (revision 247448) +++ cddl/contrib/opensolaris/tools/ctf/cvt/output.c (working copy) @@ -363,6 +363,7 @@ sort_iidescs(Elf *elf, const char *file, tdata_t * for (i = 0; i < nent; i++) { GElf_Sym sym; + char *bname; iidesc_t **tolist; GElf_Sym ssym; iidesc_match_t smatch; @@ -377,6 +378,8 @@ sort_iidescs(Elf *elf, const char *file, tdata_t * switch (GELF_ST_TYPE(sym.st_info)) { case STT_FILE: + bname = strrchr(match.iim_name, '/'); + bname = bname == NULL ? match.iim_name : bname + 1; match.iim_file = match.iim_name; continue; case STT_OBJECT: --------------060403060209040404050200-- From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 4 13:49:55 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8B23C446 for ; Mon, 4 Mar 2013 13:49:55 +0000 (UTC) (envelope-from isabell@issyl0.co.uk) Received: from mail-vc0-f175.google.com (mail-vc0-f175.google.com [209.85.220.175]) by mx1.freebsd.org (Postfix) with ESMTP id 4B3471941 for ; Mon, 4 Mar 2013 13:49:55 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id p1so2847799vcq.34 for ; Mon, 04 Mar 2013 05:49:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:x-originating-ip:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=DioQTY+wgCO7jSel/CTscrjyiF2Xu6dBG4Yow/s5vw8=; b=MWconILzyGwbfqdEJuqlobhrGviJt9woqtuzyRCL87LCqE1zI51PHjn2+GWvL8WTf8 t0BEeGOpXs4YRuA+yh6PaQQofYUpG3ivDalEQKU7JtI4NtrXM09UjImhT05j75eMr8c3 jZ71+M/d6GWsVcZwFpaA34GzdOnibWN4iHuZigCu09ZSlVtHeXhMdqgi2v9iHHJKEFIc CKWCLLQiqvPYmAA4kLUpH32RcuOlgADd/rbl3+8vTr74PnaXtvFg3IjTiYJ4cwpn2EH1 DOs1UbTDKrmGOStEvNEhSA7neRu9w/FYDgVUAhEhHnlLBbuX5iqwt+c1QTnbB54mry6B CTAg== MIME-Version: 1.0 X-Received: by 10.220.116.5 with SMTP id k5mr7718621vcq.55.1362404988940; Mon, 04 Mar 2013 05:49:48 -0800 (PST) Sender: isabell@issyl0.co.uk Received: by 10.220.250.138 with HTTP; Mon, 4 Mar 2013 05:49:48 -0800 (PST) X-Originating-IP: [2.29.6.249] Date: Mon, 4 Mar 2013 13:49:48 +0000 X-Google-Sender-Auth: 7LX9Z-YX-bFUn-7rILEEDB9MoqY Message-ID: Subject: FreeBSD Quarterly Status Report, July-September 2012. From: Isabell Long To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQkNqJNH8DtVJqC3jkNcEmUeqqSJ7PMHmROxbnkwKpRj2ipMV5fTX+7MlcNMAm/EssG404Hi Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 13:49:55 -0000 FreeBSD Quarterly Status Report, July-September 2012. Introduction This report covers FreeBSD-related projects between July and September 2012. This is the third of the four reports planned for 2012. Highlights from this quarter include successful participation in Google Summer of Code, major work in areas of the source and ports trees, and a Developer Summit attended by over 30 developers. Thanks to all the reporters for the excellent work! This report contains 12 entries and we hope you enjoy reading it. __________________________________________________________________ Projects * FreeBSD on Altera FPGAs * Native iSCSI Target * Parallel rc.d execution FreeBSD Team Reports * FreeBSD Bugbusting Team * FreeBSD Foundation * The FreeBSD Core Team Kernel * FreeBSD on ARMv6/ARMv7 Documentation * The FreeBSD Japanese Documentation Project Ports * KDE/FreeBSD * Ports Collection Miscellaneous * FreeBSD Developer Summit, Cambridge, UK FreeBSD in Google Summer of Code * Google Summer of Code 2012 __________________________________________________________________ FreeBSD Bugbusting Team URL: http://www.FreeBSD.org/support.html#gnats URL: https://wiki.freebsd.org/BugBusting Contact: Eitan Adler Contact: Gavin Atkinson Contact: Oleksandr Tymoshenko In August, Eitan Adler (eadler@) and Oleksandr Tymoshenko (gonzo@) joined the Bugmeister team. At the same time, Remko Lodder and Volker Werth stepped down. We extend our thanks to Volker and Remko for their work in the past, and welcome Oleksandr and Eitan. Eitan and Oleksandr have been working hard on migrating from GNATS, and have made significant progress on evaluating new software, and creating scripts to export data from GNATS. The bugbusting team continue work on trying to make the contents of the GNATS PR database cleaner, more accessible and easier for committers to find and resolve PRs, by tagging PRs to indicate the areas involved, and by ensuring that there is sufficient info within each PR to resolve each issue. As always, anybody interested in helping out with the PR queue is welcome to join us in #freebsd-bugbusters on EFnet. We are always looking for additional help, whether your interests lie in triaging incoming PRs, generating patches to resolve existing problems, or simply helping with the database housekeeping (identifying duplicate PRs, ones that have already been resolved, etc). This is a great way of getting more involved with FreeBSD! Open tasks: 1. Further research into tools suitable to replace GNATS. 2. Get more users involved with triaging PRs as they come in. 3. Assist committers with closing PRs. __________________________________________________________________ FreeBSD Developer Summit, Cambridge, UK URL: https://wiki.freebsd.org/201208DevSummit Contact: Robert Watson In the end of August, there was an "off-season" Developer Summit held in Cambridge, UK at the University of Cambridge Computer Laboratory. This was a three-day event, with a documentation summit scheduled for the day before. The three days of the main event were split into three sessions, with two tracks in each. Some of them even involved ARM developers from the neighborhoods which proven to be productive, and led to further engagement between the FreeBSD community and ARM. The schedule was finalized on the first day, spawning a plethora of topics to discuss, followed by splitting into groups. A short summary from each of the groups was presented in the final session and then published at the event's home page on the FreeBSD wiki. This summit contributed greatly to arriving to a tentative plan for throwing the switch to make clang the default compiler on HEAD. This was further discussed on the mailing list, and has now happened, bringing us one big step closer to a GPL-free FreeBSD 10. As part of the program, an afternoon of short talks from researchers in the Cambridge Computer Laboratory involved either operating systems work in general or FreeBSD in particular. Robert Watson showed off a tablet running FreeBSD on a MIPS-compatible soft-core processor running on an Altera FPGA. In association with the event, a dinner was hosted by St. John's college and co-sponsored by Google and the FreeBSD Foundation. The day after the conference, a trip was organized to Bletchley Park, which was celebrating Turing's centenary in 2012. __________________________________________________________________ FreeBSD Foundation URL: http://www.freebsdfoundation.org/press/2012Jul-newsletter.shtml Contact: Deb Goodkin The Foundation hosted and sponsored the Cambridge FreeBSD developer summit in August 2012. We were represented at the following conferences: OSCON July 2012, Texas LinuxFest, and Ohio LinuxFest. We negotiated/supervised Foundation funded projects: Distributed Security Audit Logging, Capsicum Component Framework, Native iSCSI Target Scoping, and Growing UFS Filesystems Online. We negotiated, supervised, and funded hardware needs for FreeBSD co-location centers. We welcomed Kirk McKusick to our board of directors. He took over the responsibility of managing our investments. We visited companies to discuss their FreeBSD use and to help facilitate collaboration with the Project. We managed FreeBSD vendor community mailing list and meetings. We created a high quality FreeBSD 9 brochure to help promote FreeBSD. Published our semi-annual newsletter that highlighted Foundation funded projects, travel grants for developers, conferences sponsored and other ways the Foundation supported the FreeBSD Project. We hired a technical writer to help with FreeBSD marketing/promotional material. We began work on redesigning our website. __________________________________________________________________ FreeBSD on Altera FPGAs URL: http://www.cl.cam.ac.uk/research/security/ctsrd/ URL: http://www.cl.cam.ac.uk/research/security/ctsrd/cheri.html Contact: Brooks Davis Contact: Robert Watson Contact: Bjoern Zeeb In the course of developing the CHERI processor as part of the CTSRD project SRI International's Computer Science Laboratory and the University of Cambridge Computer Laboratory have developed support for a number of general purpose IP cores for Altera FPGAs including the Altera Triple Speed Ethernet (ATSE) MAC core, the Altera University Program SD Card core, and the Altera JTAG UART. We have also added support for general access to memory mapped devices on the Avalon bus via the avgen bus. We have implemented both nexus and flattened device tree (FDT) attachments for these devices. In addition to these softcore we have developed support for the Terasic multi-touch LCD and are working to provide support for the Terasic HDMI Transmitter Daughter Card. Both of these work with common development and/or reference boards for Altera FPGAs. They do require additional IP cores which we plan to release to the open source community in the near future. With exception of the ATSE and HDMI drivers we have merged all of these changes to FreeBSD-CURRENT. We anticipate that these drivers will be useful for users who with to run FreeBSD on either hard or soft core CPUs on Altera FPGAs. This work has been sponsored by DARPA, AFRL, and Google. __________________________________________________________________ FreeBSD on ARMv6/ARMv7 Contact: freebsd-arm mailing list Support for ARMv6 and ARMv7 architecture has been merged from project branch to HEAD. This code covers the following parts: * General ARMv6/ARMv7 kernel bits (pmap, cache, assembler routines, etc...) * ARM Generic Interrupt Controller driver * Improved thread-local storage for cpus >=ARMv6 * Driver for SMSC LAN95XX and LAN8710A ethernet controllers * Marvell MV78x60 support (multiuser, ARMADA XP kernel config) * TI OMAP4 and AM335x support (multiuser, no GPU or graphics support, kernel configs for Pandaboard and Beaglebone) * LPC32x0 support (multiuser, frame buffer works with SSD1289 LCD controller. Embedded Artists EA3250 kernel config) This work was a result of a joint effort by many people, including but not limited to: Grzegorz Bernacki (gber@), Aleksander Dutkowski, Ben R. Gray (bgray@), Olivier Houchard (cognet@), Rafal Jaworowski (raj@) and Semihalf team, Tim Kientzle (kientzle@), Jakub Wojciech Klama (jceel@), Ian Lepore (ian@), Warner Losh (imp@), Damjan Marion (dmarion@), Lukasz Plachno, Stanislav Sedov (stas@), Mark Tinguely and Andrew Turner (andrew@). Thanks to all, who contributed by submitting code, testing and giving valuable advice. Open tasks: 1. More hardware bring-ups and more drivers 2. Finish SMP support 3. VFP/NEON support __________________________________________________________________ Google Summer of Code 2012 URL: http://www.freebsd.org/projects/summerofcode.html URL: https://wiki.freebsd.org/SummerOfCode2012 Contact: FreeBSD Summer of Code Administrators Over the Summer of 2012, FreeBSD were once again granted a place to participate in the Google Summer of Code program. We received a total of 32 project proposals, and were ultimately given 15 slots for university students to work on open source projects mentored by existing FreeBSD developers. We were able to accept a wide spread of proposals, covering both the base system and the ports infrastructure. We had students working on file systems, file integrity checking, and parallelization in the ports collection. Students worked on kernel infrastructure, including one project to support CPU resource limits on users, processes and jails, and one student improving the BSD callout(9) and timer facilities. Two students worked on the ARM platform, widely used in embedded systems and smart phones; one student worked on a significant cleanup and improvements to the Flattened Device Tree implementation code, while the other ported FreeBSD to the OMAP3-based BeagleBoard-xM device. One student worked on improving IPv6 support in userland tools, whilst another worked on BIOS emulation for the BHyVE BSD-licensed hypervisor, new in FreeBSD 10. Other students worked on EFI boot support, userland lock profiling and an automated kernel crash reporting system. Overall, a significant proportion of the code produced has or will be integrated into FreeBSD in one form or another. All of the work is available in our Summer Of Code Subversion repository, and some of the work has already been merged back into the main repositories. FreeBSD is once again grateful to Google for being selected to participate in Summer of Code 2012. __________________________________________________________________ KDE/FreeBSD URL: http://FreeBSD.kde.org URL: http://FreeBSD.kde.org/area51.php Contact: KDE FreeBSD The KDE/FreeBSD team have continued to improve the experience of KDE software and Qt under FreeBSD. The latest round of improvements include: * Fixes for building Qt with libc++ and C++11 * Fixes for Solid-related crashes * Fix battery detection in battery monitor plasmoid The team has also made many releases and upstreamed many fixes and patches. The latest round of releases include: * KDE SC: 4.9.1 (area51) and 4.8.4 (ports) * Qt: 4.8.3 (area51) * PyQt: 4.9.4 (area51); QScintilla 2.6.2 (area51); SIP: 4.13.3 (area51) * Calligra: 2.4.3, 2.5-RC2, 2.5.0. 2.5.1, 2.5.2 (area51) and 2.4.3, 2.5.0, 2.5.1 (ports) * Amarok: 2.6.0 (area51) * CMake: 2.8.9 (ports) * Digikam (and KIPI-plugins): 2.7.0, 2.8.0, 2.9.0 (area51) and 2.7.0, 2.9.0 (ports) * QtCreator: 2.6.0-beta (area51) * many smaller ports The team is always looking for more testers and porters so please contact us at kde@FreeBSD.org and visit our home page at http://FreeBSD.kde.org. Open tasks: 1. Please see 2012 Q4 Status Report 2. Updating out-of-date ports, see PortScout for a list __________________________________________________________________ Native iSCSI Target Contact: Edward Tomasz Napieral/a During the July-September time period, the Native iSCSI Target project was officially started under sponsorship from the FreeBSD Foundation. Before the end of September I've written ctld(8), the userspace part of the target, responsible for handling configuration, accepting incoming connections, performing authentication and iSCSI parameter negotiation, and handing off connections to the kernel. For the time being, I've reused some parts of protocol-handling code from the istgt project; since ctld(8) only handles the Login phase, the code can be rewritten in a much simpler and shorter way in the future. __________________________________________________________________ Parallel rc.d execution URL: https://github.com/buganini/rcexecr URL: https://github.com/kil/rcorder Contact: Kuan-Chung Chiu Contact: Kilian There are two implementations to make rc.d execution parallel. Compared to Kil's rcorder, rcexecr brings more concurrence and provides more flexibility than older "early_late_divider" mechanism but require more invasive /etc patch. Both implementations have switch to toggle parallel execution. Further modification/integration needs more discussion. Open tasks: 1. Refine /etc/rc.d/* to eliminate unnecessary waiting. __________________________________________________________________ Ports Collection URL: http://www.FreeBSD.org/ports/ URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing-ports/ URL: http://portsmon.freebsd.org/index.html URL: http://www.freebsd.org/portmgr/index.html URL: http://blogs.freebsdish.org/portmgr/ URL: http://www.twitter.com/freebsd_portmgr/ URL: http://www.facebook.com/portmgr Contact: Thomas Abthorpe Contact: Port Management Team The ports tree approaches 24,000 ports, while the PR count still is above 1000. In Q3 we added 2 new committers and took in two commits bit for safe keeping. The Ports Management team had performed multiple -exp runs, verifying how base system updates may affect the ports tree, as well as providing QA runs for major ports updates. Beat Gaetzi took over the role of sending out fail mails, a role that Pav Lucistnik had previously held. Beat also undertook the task of converting the Ports tree from CVS to Subversion. Florent Thoumie stepped down from his role on portmgr, he was instrumental in maintaining the legacy pkg_* code. Open tasks: 1. Most ports PRs are assigned, we now need to focus on testing, committing and closing. __________________________________________________________________ The FreeBSD Core Team Contact: Core Team Along with the change in the Core Team membership, several related roles changed hands. Gabor Pali assumed the role of core secretary from Gavin Atkinson, and David Chisnall replaced Robert Watson as liaison to the FreeBSD Foundation. The Core Team felt there was no longer a need for a formal security team liaison, so that role was retired. In the third quarter, the Core Team granted access for 2 new committers and took 2 commit bits into safekeeping. The Core Team worked with the Port Management Team and Cluster Administrators to set a date to stop providing CVS exports for the ports repository, which is February 28, 2013. In the meantime, the CVS export for 9.1-RELEASE was restored. __________________________________________________________________ The FreeBSD Japanese Documentation Project URL: http://www.FreeBSD.org/ja/ URL: http://www.jp.FreeBSD.org/doc-jp/ Contact: Hiroki Sato Contact: Ryusuke Suzuki Web page (htdocs): Newsflash and some other updates in the English version were translated to keep them up-to-date. Especially "security incident on FreeBSD infrastructure" was translated and published in a timely manner. FreeBSD Handbook: Big update in the "advanced-networking". With this update, merging translation results from the handbook in the local repository of Japanese documentation project into the main repository was completed. This chapter is still outdated and needs more work. The other sections have also constantly been updated. Especially, new subsection "Using pkgng for Binary Package Management" was added to "ports" section and "Using subversion" subsection was added to "mirrors" section. Article: Some progress was made in "Writing FreeBSD Problem Reports" and "Writing FreeBSD Problem Reports" articles. Open tasks: 1. Further translation work of outdated documents in the ja_JP.eucJP subtree. __________________________________________________________________ From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 4 13:54:05 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 11ECC67B for ; Mon, 4 Mar 2013 13:54:05 +0000 (UTC) (envelope-from isabell@issyl0.co.uk) Received: from mail-vc0-f179.google.com (mail-vc0-f179.google.com [209.85.220.179]) by mx1.freebsd.org (Postfix) with ESMTP id CB4B41998 for ; Mon, 4 Mar 2013 13:54:04 +0000 (UTC) Received: by mail-vc0-f179.google.com with SMTP id k1so3454943vck.10 for ; Mon, 04 Mar 2013 05:54:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:x-originating-ip:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding:x-gm-message-state; bh=TAF5gStlzh2lRTRynJnPHC8g/JD6grypvGawb0QOpBE=; b=CZ4j/wk6NrdSvow6Ig+N4nwei4f2K68MvWriU2KgKKGxLh5VlIBcwMmKHDZy0nPMhU 84Oqf5WZr5k8SGO63gOBVc/PA4muOEZDTVIcVDT9GjTfvJf64ofP3rqjMdQ7WCkb22j+ bFDbkqRhabHQruOFfj0lStAAbYaDPDnKDq5rvXMOI2aKR8oHajlvVy7jbQjr8Y3dH7R4 f15QRaoV0baWW4Vkyv18L8nWwvDE125l4sey0Ki31k40TOb5e4oPrE+YlECSBiOx0t3t cojfz0EIZDjBk8FSWHjLGf8M9tUbqalCjamP7EYhFOa24HDohNyQ/ZvuHgGQllD8v7Dp 0IFA== MIME-Version: 1.0 X-Received: by 10.220.242.73 with SMTP id lh9mr7566953vcb.49.1362405243743; Mon, 04 Mar 2013 05:54:03 -0800 (PST) Sender: isabell@issyl0.co.uk Received: by 10.220.250.138 with HTTP; Mon, 4 Mar 2013 05:54:03 -0800 (PST) X-Originating-IP: [2.29.6.249] Date: Mon, 4 Mar 2013 13:54:03 +0000 X-Google-Sender-Auth: OHpE0Y1ylrkCvNGJ-qAPGcvxdCs Message-ID: Subject: FreeBSD Quarterly Status Report, October-December 2012. From: Isabell Long To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQkCXRHMLLNMHIaNhHEGTmlO/oQOFGtLwiD1mdG82z1J0Z0kuVkFekCj0aE+wGq2Hd78HsZ0 Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 13:54:05 -0000 FreeBSD Quarterly Status Report, October-December 2012. Introduction This report covers FreeBSD-related projects between October and December 2012. This is the last of four reports planned for 2012. Highlights from this status report include a very successful EuroBSDCon 2012 conference and associated FreeBSD Developer Summit, both held in Warsaw, Poland. Other highlights are several projects related to the FreeBSD port to the ARM architecture, extending support for platforms, boards and CPUs, improvements to the performance of the pf(4) firewall, and a new native iSCSI target. Thanks to all the reporters for the excellent work! This report contains 27 entries and we hope you enjoy reading it. The deadline for submissions covering the period between January and March 2013 is April 21st, 2013. __________________________________________________________________ Projects * BHyVe * Native iSCSI Target * NFS Version 4 * pxe_http -- booting FreeBSD from apache * UEFI * Unprivileged install and image creation Userland Programs * BSD-licenced patch(1) * bsdconfig(8) FreeBSD Team Reports * FreeBSD Core Team * FreeBSD Documentation Engineering * FreeBSD Foundation * Postmaster Kernel * AMD GPUs kernel-modesetting support * Common Flash Interface (CFI) driver improvements * SMP-Friendly pf(4) * Unmapped I/O Documentation * The FreeBSD Japanese Documentation Project Architectures * Compiler improvements for FreeBSD/ARMv6 * FreeBSD on AARCH64 * FreeBSD on BeagleBone * FreeBSD on Raspberry Pi Ports * FreeBSD Haskell Ports * KDE/FreeBSD * Ports Collection * Xfce Miscellaneous * EuroBSDcon 2012 * FreeBSD Developer Summit, Warsaw __________________________________________________________________ AMD GPUs kernel-modesetting support URL: https://wiki.FreeBSD.org/AMD_GPU URL: http://people.FreeBSD.org/~kib/misc/ttm.1.patch Contact: Alexander Kabaev Contact: Jean-S=E9bastien P=E9dron Contact: Konstantin Belousov Jean-S=E9bastien P=E9dron started to port the AMD GPUs driver from Linux= to FreeBSD 10-CURRENT in January 2013. This work is based on a previous effort by Alexander Kabaev. Konstantin Belousov provided the initial port of the TTM memory manager. As of this writing, the driver is building but the tested device fails to attach. Status updates will be posted to the FreeBSD wiki. __________________________________________________________________ BHyVe URL: https://wiki.FreeBSD.org/BHyVe URL: http://www.bhyve.org/ Contact: Neel Natu Contact: Peter Grehan BHyVe is a type-2 hypervisor for FreeBSD/amd64 hosts with Intel VT-x and EPT CPU support. The bhyve project branch was merged into CURRENT on Jan 18. Work is progressing on performance, ease of use, AMD SVM support, and being able to run non-FreeBSD operating systems. Open tasks: 1. 1. Booting Linux/*BSD/Windows 2. 2. Moving the codebase to a more modular design consisting of a small base and loadable modules 3. 3. Various hypervisor features such as suspend/resume/live migration/sparse disk support __________________________________________________________________ BSD-licenced patch(1) URL: http://code.google.com/p/bsd-patch/ Contact: Pedro Giffuni Contact: Gabor Kovesdan Contact: Xin Li FreeBSD has been using for a while a very old version of GNU patch that is partially under the GPLv2. The original GNU patch utility is based on an initial implementation by Larry Wall that was not actually copyleft. OpenBSD did many enhancements to an older non-copyleft version of patch, this version was later adopted and further refined by DragonFlyBSD and NetBSD but there was no centralized development of the tool and FreeBSD kept working independently. In less than a week we took the version in DragonFlyBSD and adapted the FreeBSD enhancements to make it behave nearer to the version used natively in FreeBSD. Most of the work was done by Pedro Giffuni, adapting patches from sepotvin@ and ed@, and additional contributions were done by Christoph Mallon, Gabor Kovesdan and Xin Li. As a result of this we now have a new version of patch committed in head/usr.bin/patch that you can try by using WITH_BSD_PATCH in your builds. The new patch(1) doesn't support the FreeBSD-specific -I and -S options which don't seem necessary. In GNU patch -I actually means 'ignore whitespaces' and we now support it too. Open tasks: 1. Testing. A lot more testing. __________________________________________________________________ bsdconfig(8) URL: http://svnweb.FreeBSD.org/base/head/usr.sbin/bsdconfig/ URL: http://freshports.org/sysutils/bsdconfig/ URL: http://druidbsd.sf.net/download/bsdconfig/ Contact: Devin Teske bsdconfig(8) is actively being developed in HEAD under the WITH_BSDCONFIG build-requirement. Snapshots are occasionally taken and made available through the ports system to make testing on 9.0-RELEASE or higher easier on the testers. Currently HEAD is far beyond the version 0.7.3 sitting in ports. Upcoming changes will push this to version 0.8 bringing in the necessary frameworks required for in-depth package management and distribution maintenance (read: one step closer to full 1.0 release). __________________________________________________________________ Common Flash Interface (CFI) driver improvements Contact: Brooks Davis The Common Flash Interface provides a common programming interface for a wide range of NOR flash devices commonly found in embedded systems. I have developed a number of improvements to the cfi(4) device when used on Intel StrataFlash parts. Unnecessary erase cycles are now avoided, devices that require single word writes only write changed words, and multi-word writes are supported for Intel and Sharp devices. Additionally the timeout code has been reworked and no longer imposes unneeded latency on operations taking less than 100us. With all of these changes streaming write speed has improved by more than an order of magnitude. Once these changes are reviewed they will be committed to HEAD. This work was sponsored by DARPA and AFRL. __________________________________________________________________ Compiler improvements for FreeBSD/ARMv6 Contact: Andrew Turner FreeBSD/ARM architecture is now supported by the in-tree clang compiler. ARM EABI support is now available for both clang and gcc along with the older and less documented OABI. There are several outstanding issues, once they are fixed EABI will be made default. Open tasks: 1. Test EABI builds 2. Fix exception handling for EABI 3. Test clang builds 4. Get clang to work natively on EABI-based ARM system. Currently it works only as cross-compiler for ARM EABI. __________________________________________________________________ EuroBSDcon 2012 URL: http://2012.eurobsdcon.org/ URL: http://www.youtube.com/user/eurobsdcon Contact: EuroBSDcon Organizers Contact: Gabor Pali The 11th European BSD Conference took place in Warsaw, Poland at the Warsaw University of Technology with a large number of visitors. It started up with two tracks of tutorials, featuring FreeNAS, pfSense, DTrace, PF, development of NetBSD drivers, and an overall introduction to the FreeBSD operating system given by Kirk McKusick. There we also had opening and closing keynotes, supplemented with 22 talks on different topics related to FreeBSD, OpenBSD, NetBSD, FreeNAS and PC-BSD: BHyVe, configuration management with puppet, improvements in the OpenBSD cryptographic framework, tuning ZFS, server load balancing in DNS, running FreeBSD on embedded systems, e.g MIPS and ARM, and challenges in identity management and authentication. The conference also had a dedicated track presented by the attendees of the FreeBSD developer summit and open to all, where one could learn more about what is happening currently in the Project: results of Google Summer of Code 2012, architectural changes in the FreeBSD documentation tree, ILNP, advancements in package building and development of pkg(8), and a status report on the USB stack. __________________________________________________________________ FreeBSD Core Team Contact: Core Team In the fourth quarter, the Core Team granted access for 7 new committers, and took 1 commit bit in for safekeeping. The Core Team oversaw the response to the security incident in November in cooperation with the security team, port managers, and cluster administrators. For more information on the fallouts and response see the official announcement. As a result, 9.1-RELEASE was delayed until late December and was released with a limited set of binary packages. The Core Team continues to work with developers to rebuild, review, and restore the package building infrastructure along with redports/QAT. __________________________________________________________________ FreeBSD Developer Summit, Warsaw URL: http://wiki.FreeBSD.org/201210DevSummit Contact: Gabor Pali We had 53 FreeBSD developers and invited guests attending the FreeBSD Developer Summit organized as part of EuroBSDcon 2012 in Warsaw, Poland at the Warsaw University of Technology. This year EuroBSDcon organizers again offered us their generous support in helping with keeping the event running smooth, helping with registrations, renting the venue, and providing food for keeping attendees satisfied and happy. The Warsaw developer summit spanned over 3 days and had 9 working groups on various topics. We improved last year's layout inherited from the Canadian summits because it has worked well earlier but could use some further refinements. On both the first and second days, we ran the working groups, ranging from the standard matters, discussing issues with the USB stack, the compiler toolchain, the Ports Collection, or the documentation to some experimental ones, e.g. arranging an operating systems course focusing on FreeBSD. In addition to this, similarly to last year, one of the working groups was about gathering vendors to present their ideas and engage in discussion with the developers on their needs from the Project. Finally, on the third day, there were a number of exciting work-in-progress reports given in a dedicated Developer Summit track at the main conference. Photos and slides for the most of the talks are available on the home page of the summit. __________________________________________________________________ FreeBSD Documentation Engineering URL: http://www.FreeBSD.org/internal/doceng.html Contact: Glen Barber Contact: Marc Fonvieille Contact: G=E1bor K=F6vesd=E1n Contact: Hiroki Sato The translations/, projects/ and user/ directories of the doc repository have been opened with the announced policies in effect. These branches are now actively used for translations work, editing the upcoming printed version of the Handbook, and some doc infrastructure improvements. The next phase of the infrastructure improvements is in progress. It will migrate to real XML tools (with the exception of Jade) for validation and rendering. At the same time, the DocBook schema will be updated to 4.5. After long discussions, Google Analytics has been enabled on FreeBSD.org webpages but access to statistical data has to be solicited from the Documentation Engineering Team on an individual and one time basis. Since July, we have added two doc committers and one translator. Open tasks: 1. Help the ongoing work on printed edition of the Handbook. 2. Finish the migration to XML tools. __________________________________________________________________ FreeBSD Foundation Contact: Deb Goodkin A strong year-end fundraising campaign led to the raising $770,000 in 2012. Thank you to everyone who made a donation to support FreeBSD! We published our year-end newsletter that highlighted everything we did to support the FreeBSD Project and community during the second half of the year. We were a Gold Sponsor for EuroBSDCon. We also attended the conference and developer summit. Erwin Lansing organized and chaired the Ports and Package Summit and Vendor Summit at EuroBSDCon 2012. We attended MeetBSD developer summit November 2012. George Neville-Neil organized and the Foundation sponsored the Bay Area Vendor Summit November 2012. We were represented at LISA. Kirk McKusick taught a tutorial and gave a keynote at EuroBSDCon 2012, and Justin Gibbs gave a talk at ZFS Day, October 2012. We talked to DNS server software vendors and participated in discussions on our DNS implementation, specifically with regard to DNSSEC validation, at CENTR Tech September 2012 (Amsterdam, the Netherlands) and EuroBSDCon. We visited companies to discuss their FreeBSD use and to help facilitate collaboration with the Project. Robert Watson published ACM Queue and Communications of the ACM: A decade of OS access-control extensibility and Kirk McKusick published ACM Queue and Communications of the ACM: Disks from the Perspective of a File System. We negotiated/supervised Foundation funded projects: porting FreeBSD to the Efika ARM platform, Capsicum Component Framework, Native iSCSI Target implementation, and EUFI. We negotiated/supervised/funded hardware needs in FreeBSD co-location centers. Many board members provided support for recovery efforts following the security compromise of FreeBSD.org systems in late 2012. We completed negotiation and provided legal counsel for the new website privacy policy for the FreeBSD Project. We are now an industrial partner in the Cambridge/Imperial/Edinburgh EPSRC REMS project on the Rigorous Engineering of Mainstream Systems. We coordinated the Foundation's discussion of Jira/Java; conclusion, will continue to be supportive of OpenJDK and not restart proprietary JDK support. We implemented a donor management database to help with our fundraising efforts. We also began working on automating the donation process. We started the Faces of FreeBSD Series where we share the story of a Foundation grant recipient periodically. This allows us to spotlight people who received Foundation funding to work on development projects, run conferences, travel to conferences, and advocate for FreeBSD. We hired two technical staff members. __________________________________________________________________ FreeBSD Haskell Ports URL: http://wiki.FreeBSD.org/Haskell URL: https://github.com/FreeBSD-haskell/FreeBSD-haskell/ Contact: G=E1bor P=C1LI Contact: Ashish SHUKLA We are proud to announce that the FreeBSD Haskell Team has updated the Haskell Platform to 2012.4.0.0, GHC to 7.4.2 as well as updated existing ports to their latest stable versions. All Haskell ports are also updated to use new OPTIONS framework, and now, building with dynamic libraries (DYNAMIC) is on by default. GHC also uses GCC 4.6 and binutils 2.22 from ports. We also added a number of new Haskell ports, and their count in FreeBSD Ports tree is now 368. Open tasks: 1. Test GHC to work with clang/LLVM. 2. Commit pending Haskell ports to the FreeBSD Ports tree. 3. Add more ports to the Ports Collection. __________________________________________________________________ FreeBSD on AARCH64 URL: https://github.com/zxombie/aarch64-freebsd-sandbox URL: http://www.arm.com/products/tools/models/fast-models/foundation-model.p hp Contact: Andrew Turner Work has started on porting FreeBSD to AARCH64, ARM's new 64-bit architecture, using the ARMv8 Foundation Model software. GCC and binutils have been ported to FreeBSD and work started on kernel initialization, including MMU setup. Open tasks: 1. Get the MMU working 2. Get system register documentation from ARM 3. Port clang AArch64 to FreeBSD 4. Bring the code into a FreeBSD project branch __________________________________________________________________ FreeBSD on BeagleBone Contact: Tim Kientzle Contact: Oleksandr Tymoshenko Contact: Damjan Marion Contact: Brett Wynkoop FreeBSD on BeagleBone is benefiting from the general work on ARM stability being done by many people, and is proving to be a nice testbed for our ARMv7 support. All ongoing work is happening now directly in -CURRENT and we expect it to be in pretty good shape by the time 10.0 ships. The network driver is now pretty stable; the system should be useful as a small network device. Occasional system snapshots are being built and advertised for people to test. Ask on freebsd-arm@ if you'd like to try the newest one. Open tasks: 1. We need someone to finish the USB driver. Ask if you'd like to take this over. 2. MMCSD performance is still rather poor. 3. There's been discussion of how to improve the GPIO configuration and pinmux handling to simplify hardware experimentation. If we had more people to help build drivers, we could start supporting some of the BeagleBone capes. 4. Mostly we just need people to use it and report any issues they encounter. __________________________________________________________________ FreeBSD on Raspberry Pi Contact: Oleksandr Tymoshenko FreeBSD is running on Raspberry Pi and supports the following peripherals: * USB controller * SDHC controller * Network * Framebuffer (HDMI and composite) * GPIO * VCHI interface Videocore tests (OpenGL, video decoding, audio, display access) work with current VCHI driver implementation. Open tasks: 1. Add DMA mode support to USB driver. Some proof-of-concept code is done but more work required to finish it. 2. Re-implement VCHI driver with more FreeBSD-friendly locking. 3. Implement more drivers: SPI, PWM, audio. __________________________________________________________________ Google Summer of Code 2013 URL: http://www.FreeBSD.org/projects/summerofcode.html URL: http://google-opensource.blogspot.co.uk/2013/02/flip-bits-not-burgers-g oogle-summer-of.html URL: http://www.google-melange.com/gsoc/homepage/google/gsoc2013 URL: http://en.wikipedia.org/wiki/Google_Summer_of_Code Contact: FreeBSD Summer of Code Administrators Since 2005 Google has run its yearly Summer of Code program, in which Google awards stipends to students who successfully complete projects with participating Open Source organisations. FreeBSD has participated in GSoC every year since its inception, and with the announcement that Google will once again run the program in 2013 hopes to participate once more. Google have not yet opened the application period for mentoring organisations, but once it does FreeBSD plans to apply. Assuming that we are successful in our application to participate, we will publish a large list of ideas for possible projects shortly after. Students may then apply to do one of those projects, or suggest their own idea for a project. After the application period, FreeBSD will discover how many student slots we have been allocated, at which point successful students will take some time to plan their project, gather required information and discuss their plans with their mentors, before having around 12 weeks to develop their code. In the eight years of FreeBSD's participation in Google Summer of Code, approximately 150 students have successfully complete projects with us, covering a wide spread of areas of both the source and ports trees. Of these, 22 students continued participating with FreeBSD and subsequently became full FreeBSD committers, many later going on to mentor Summer of Code students themselves. Whether FreeBSD has been successful in being selected to be a participating organisation in Google Summer of Code 2013 should be announced in early April. __________________________________________________________________ KDE/FreeBSD URL: http://FreeBSD.kde.org URL: http://FreeBSD.kde.org/area51.php Contact: KDE FreeBSD The KDE/FreeBSD team have continued to improve the experience of KDE software and Qt under FreeBSD. The latest round of improvements include: * Fix handling of Removable property in solid engine * Fix management of backlight with UPower (requires acpi_video(4)) * Installing spell-checking dictionaries with a dependency of KDE-locale ports The team has also made many releases and upstreamed many fixes and patches. The latest round of releases include: * KDE SC: 4.9.2 (area51) * PyQt: 4.9.5 (area51); SIP: 4.14 (area51) * KDevelop: 4.4.0, 4.4.1 (area51); KDevPlatform: 1.4.0, 1.4.1 (area51) * Calligra: 2.5.3, 2.5.4 (area51) * CMake: 2.8.10.1 * Many smaller ports The team is always looking for more testers and porters so please contact us at kde@FreeBSD.org and visit our home page at http://FreeBSD.kde.org. Open tasks: 1. Updating out-of-date ports, see PortScout for a list __________________________________________________________________ Native iSCSI Target Contact: Edward Tomasz Napieral/a During the October-December time period, the Native iSCSI Target project progressed to the working prototype stage. Most of this time was spent writing kernel-based part, an iSCSI frontend to the CAM Target Layer. The frontend handles iSCSI Full Feature phase after ctld(8) hands off the connection. The istgt-derived code in ctld(8) was rewritten from scratch; now it's much shorter and more readable. The ctladm(8) utility gained iSCSI-specific subcommands to handle tasks such as listing iSCSI sessions or forcing disconnection. The target works correctly with the FreeBSD initiator. __________________________________________________________________ NFS Version 4 Contact: Rick Macklem The NFSv4.1 client, including support for pNFS for the Files Layout only, has now been committed to head/current. Work on NFSv4.1 server support has just been started and will hopefully be ready for head/current this summer. The client side disk caching of delegated files is progressing and the code is under projects/nfsv4-packrats in the subversion repository. Someone is working on server side referrals and, as such, I hope this might make it into 10.0 as well. __________________________________________________________________ Ports Collection URL: http://www.FreeBSD.org/ports/ URL: http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributing-ports/ URL: http://portsmon.FreeBSD.org/index.html URL: http://www.FreeBSD.org/portmgr/index.html URL: http://blogs.FreeBSDish.org/portmgr/ URL: http://www.twitter.com/freebsd_portmgr/ URL: http://www.facebook.com/portmgr Contact: Thomas Abthorpe Contact: Port Management Team The ports tree crossed the threshold of 24,000 ports, while the PR count still is close to 1600. In Q4 we added five new committers and took in two commit bits for safe keeping. In the tradition of recruiting new portmgr@ at conferences, we added Bernhard Froehlich to our ranks. He is the one responsible for redports.org Pav Lucistnik stepped down from his role on portmgr, he was one of our principles doing -exp runs and well known for sending failmails. In the well publicised compromise, the pointyhat machines were broken into and subsequently taken down, isolated and sanitised. As a pre-emptive move redports/QAT were also taken down. Work is under way to restore the services. Mark Linimon began a from-scratch test install on one of his own spare machines with the purpose of documenting all the missing steps from the portbuild article. While doing so, he further overhauled the codebase to both make it easier to install, and to further refactor it in light of a security review (still ongoing at time of this writing). Once this is complete, the next task will be to reinstall all existing machines from scratch. Open tasks: 1. Most ports PRs are assigned, we now need to focus on testing, committing and closing. __________________________________________________________________ Postmaster Contact: David Wolfskill The postmaster team has expanded, with the addition of Florian Smeets (flo@FreeBSD.org). We have implemented a Mailman "handler" to drop duplicate messages when both copies are sent to the same list (under both the "long" (e.g., "freebsd-current") and "short" (e.g., "current") names). We have created several new mailing lists: * freebsd-course: educational course on FreeBSD * freebsd-numerics: Discussions of high quality implementation of libm functions. * freebsd-snapshots: FreeBSD Development Snapshot Announcements * freebsd-tcltk: FreeBSD-specific Tcl/Tk discussions We have also removed old mailing lists: * freebsd-binup * freebsd-www (merged into freebsd-doc) __________________________________________________________________ pxe_http -- booting FreeBSD from apache URL: http://svnweb.FreeBSD.org/base/user/sbruno/pxe_http_head/ Contact: Sean Bruno Currently works with VirtualBox VMs and Apache 2.2 port. Open tasks: 1. Lots and lots of compile warnings exist with clang and gcc. This really needs to be investigated. 2. Better support for other webservers. Currently needs Apache to work. 3. Needs another pass at basic documentation. Current documentation is actually quite good from the original 4. Network stack needs audit. I'm not sure if the HTTP/TCP/UDP/IP code is original or based on something else. __________________________________________________________________ SMP-Friendly pf(4) Contact: Gleb Smirnoff The project is aimed at moving the pf(4) packet filter out of a single mutex, as well as in general improving of the FreeBSD port. The project has reached its main goal. The pf(4) is no longer covered by single mutex and contention on network stack on pf(4) is now very low. The code is production ready. The projects/pf branch had been merged to the head branch and will be available in 10.0-RELEASE. __________________________________________________________________ The FreeBSD Japanese Documentation Project URL: http://www.FreeBSD.org/ja/ URL: http://www.jp.FreeBSD.org/doc-jp/ Contact: Hiroki Sato Contact: Ryusuke Suzuki The ja_JP.eucJP subtree has constantly been updated since the last status report. In FreeBSD Handbook, translation work of the "users" section has been completed. "linuxemu" and "serialcomms" were updated and subsection "Subversion mirror site" was newly added to "mirrors" section. Open tasks: 1. Further translation work of outdated documents in the ja_JP.eucJP subtree. __________________________________________________________________ UEFI URL: https://wiki.FreeBSD.org/UEFI URL: http://svnweb.FreeBSD.org/base/projects/uefi/ Contact: Benno Rice There is code in the projects/uefi branch that can build a working 64-bit loader for UEFI. This loader can load a kernel and boot to a mountroot prompt on a serial console on a system with <=3D 1GB of RAM. Full multiuser has not yet been tested. Work is progressing towards having a working syscons. The issue preventing boot on systems with > 1GB of RAM has not yet been found. UEFI-compatible boot media can be generated using in-tree tools, however there are issues with detecting the CD filesystem and using it as the load default. The 64-bit UEFI loader can load a 32-bit kernel but currently cannot hand over to it due to a lack of code to switch to 32-bit mode. Further research is required into Secure Boot. __________________________________________________________________ Unmapped I/O URL: http://people.FreeBSD.org/~kib/misc/unmapped.13.patch Contact: Jeff Roberson Contact: Konstantin Belousov A well-known performance problem of FreeBSD on large SMP hardware is the need to invalidate TLB for all CPUs when instantiating and destroying the VMIO buffers. Invalidation is performed by sending inter-processor interrupt broadcast, which disrupts the execution path of each CPU, and induces latency on the request itself. Since most I/O requests processing require creation of the buffers to hold the data in the kernel, TLB invalidation becomes an obstacle for I/O scalability on many-CPU machines. The work done for flushing the TLBs is especially meaningless since most mappings created are not used for anything but copying the data from the usermode to the kernel page cache forth and back. Most architectures have already established facilities to perform such copies using much faster techniques, for instance, the direct map on amd64, or specially reserved per-CPU page frames or TLB entries on other architectures. Jeff Roberson unified the machine-specific parts of the busdma(9), making a common set of low-level functions available on each architecture. This was committed as r246713. The end result is that the new types of the load functions can be added in the single, machine-independent place. In particular, it is easy to modify the drivers to accept the 'unmapped' bio requests, which lists the vm pages for the device dma engine, instead of the virtual address of the kernel buffer. Konstantin Belousov developed the changes for buffer cache which allow the VMIO buffers to not map the referenced pages, and used the feature for UFS. Per-architecture pmap_copy_pages(9) methods were added to facilitate fast copying between user I/O buffers and pages of unmapped buffers. The unmapped buffers create the unmapped bio requests for the drivers, support for which was made possible by Jeff's patch. Tests show that even on a small 4-core machine, the system time for reading files on UFS is reduced by 30%. Open tasks: 1. Test the patch, in particular, on non-x86 architectures. __________________________________________________________________ Unprivileged install and image creation Contact: Brooks Davis In order to make it easier to build releases and embedded system disk images I have been adding infrastructure to allow the install and packaging stages to the FreeBSD build progress to run without root privilege. To this end I have added two options to the toplevel build system: The -DDB_FROM_SRC option allows the install to proceed when the required set of passwd and group entires does not match the host system. The -DNO_ROOT option causes files to be installed as the running user and for metadata such as owner, group, suid bits, and file flags to be logged in a ${DESTDIR}/METALOG file. This work required the import of NetBSD's mtree and the addition of a number of features from NetBSD to install. I have added all FreeBSD features to NetBSD's mtree and imported it as nmtree. Before FreeBSD 10.0 is released I will replace our version. I have also added all required features to install. Changes to makefs were required to parse the contents of the METALOG file. These new features required importing new versions of the pwcache(3) and vis(3) APIs from NetBSD so those portions of libc. In addition to modifying build infrastructure to use the new features of mtree and install. I corrected a number of cases of files being installed by programs other than install or being installed more than once. A few known instances of duplicate directories in the output exist, but the results are usable in some contexts. I plan to MFC these changes as far back as the stable/8 branch to make it possible to build all supported releases without root privilege. This work was sponsored by DARPA and AFRL. Open tasks: 1. Add support for -DNO_ROOT to src/release/Makefile so that releases can be built without root privilege. 2. Create a tool to install partition tables and file system images in disk image files without the use of mdctl, gpart, and dd. __________________________________________________________________ Xfce URL: https://wiki.FreeBSD.org/Xfce Contact: A major update has been made to Thunar (file manager for the Xfce Desktop Environment). 1.6.x series introduce lots of improvements, most noticeably is tabs support, and the performance has been improved. Open tasks: 1. Try to fix HSTS (HTTP Strict Transport Security) feature in Midori with Vala 0.12.1 (works fine with Vala >=3D 0.14.x) 2. Replace libxfce4gui (deprecated and not maintained by upstream) by libxfce4ui in order to enhance support for Xfce >=3D 4.10. 3. Test core and plugins (panel, Thunar) with GLib >=3D 4.32 (to replac= e deprecated and removed functions introduced since GLib 2.30). 4. Fix gtk-xfce-engine with Gtk+ >=3D 3.6. __________________________________________________________________ From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 4 20:58:01 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CA661522 for ; Mon, 4 Mar 2013 20:58:01 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from mailbox.talkpoint.com (mailbox.talkpoint.com [204.141.10.74]) by mx1.freebsd.org (Postfix) with ESMTP id 5A1DD1489 for ; Mon, 4 Mar 2013 20:58:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailbox.talkpoint.com (Postfix) with ESMTP id E5213F66003 for ; Mon, 4 Mar 2013 15:50:22 -0500 (EST) X-Virus-Scanned: amavisd-new at talkpoint.com Received: from mailbox.talkpoint.com ([127.0.0.1]) by localhost (mailbox.talkpoint.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qK+XDB-wVVqN for ; Mon, 4 Mar 2013 15:50:21 -0500 (EST) Received: from pleiades.nextvenue.com (pleiades.nextvenue.com [204.141.15.194]) by mailbox.talkpoint.com (Postfix) with ESMTP id 11D04F66001 for ; Mon, 4 Mar 2013 15:50:21 -0500 (EST) Date: Mon, 4 Mar 2013 15:50:20 -0500 From: Nick Evans To: hackers@freebsd.org Subject: MySQL Data_free = 0 Slowness (appending to files slow?) Message-ID: <20130304155020.77e470da@pleiades.nextvenue.com> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.6; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 20:58:01 -0000 Hackers, I ran into this issue while setting up a performance testing environment with MySQL 5.6/MyISAM tables. I have an arbitrarily sized test table, in this case with a base number of rows over 1 million. When I insert new rows into this table with super-smack, and the on disk table file is full (Data_free=0 in SHOW TABLE STATUS;), I see an average queries per second of around 1500-1600. If I then delete a subset of the table's rows, reboot the test environment, and rerun my super-smack inserts, I'll see an average qps of 8000 until the table fills up again. In this second state, the on-disk MyISAM table file is not full, ie, Data_free is some non-zero value as reported by SHOW TABLE STATUS;. I'm assuming mysqld is just seeking to a spot and writing in this case vs whatever append operation has to be done by the OS when there's no more space in the table file. I'm not entirely sure where the problem lies, but I've tried this with SU+J, SU, a MD device, async, noatime, etc and have not found a way to speed up the case where I'm appending to the table file. I've also tried dumping indexes from the table to make sure they were not interfering, no change. I have ktraces showing mysqld doing the same number of pwrite operations for my test inserts when the table file is full vs not. The test system is running 9-STABLE from last week, and is an 8 core, dual package Xeon with 24GB ram, an LSI 9260-8i with two volumes, one raid1, one raid6 both using Raptor 300's. I'm aware of raid6's write speed issues, but the limiting factor here does not appear to be the card or the disks/arrays based on how well it performs in the "fast" case. I've started to look at DTrace to maybe help diagnose the problem but I still have to learn enough of it to be useful and I can't seem to get mysql56-server to compile with DTrace. The kernel is GENERIC with DTRACE hooks added. I'd appreciate any help you guys and gals can offer. Thanks, Nick iostat (slow case): tty mfid0 mfid1 cd0 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 479 0.00 0 0.00 80.00 2 0.16 0.00 0 0.00 11 0 1 0 88 0 1723 0.00 0 0.00 128.00 2 0.25 0.00 0 0.00 11 0 0 0 89 0 442 0.00 0 0.00 128.00 2 0.25 0.00 0 0.00 10 0 1 0 88 0 642 0.00 0 0.00 80.00 2 0.16 0.00 0 0.00 11 0 1 0 88 0 608 0.00 0 0.00 96.00 3 0.28 0.00 0 0.00 11 0 0 0 88 0 681 0.00 0 0.00 64.00 3 0.19 0.00 0 0.00 11 0 1 0 88 0 1477 0.00 0 0.00 64.00 6 0.37 0.00 0 0.00 11 0 0 0 88 0 592 0.00 0 0.00 128.00 1 0.12 0.00 0 0.00 11 0 1 0 88 0 709 32.00 10 0.31 80.00 2 0.16 0.00 0 0.00 12 0 1 0 86 0 3925 0.00 0 0.00 22.00 4 0.09 0.00 0 0.00 14 0 1 0 85 0 590 8.00 1 0.01 32.00 2 0.06 0.00 0 0.00 13 0 1 0 86 0 1707 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 615 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 594 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 14 0 1 0 85 0 618 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 14 0 1 0 85 0 531 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 1640 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 13 0 0 0 86 0 497 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 554 32.00 2 0.06 127.16 757 94.03 0.00 0 0.00 14 0 2 0 84 0 590 0.00 0 0.00 117.33 24 2.75 0.00 0 0.00 14 0 1 0 85 0 533 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 13 0 1 0 85 0 1711 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 0 0 86 0 557 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 13 0 0 0 86 0 502 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 14 0 1 0 85 0 717 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 13 0 1 0 86 0 564 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 13 0 1 0 86 0 1548 0.00 0 0.00 32.00 4 0.12 0.00 0 0.00 14 0 1 0 85 0 559 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 13 0 1 0 85 0 543 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 14 0 1 0 85 0 493 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 13 0 1 0 86 0 623 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 1656 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 14 0 1 0 85 0 569 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 13 0 1 0 86 0 540 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 608 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 13 0 1 0 85 0 580 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 13 0 1 0 86 tty mfid0 mfid1 cd0 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 1644 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 13 0 1 0 85 0 752 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 575 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 13 0 1 0 86 0 498 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 14 0 1 0 85 0 495 0.00 0 0.00 18.67 3 0.05 0.00 0 0.00 13 0 1 0 86 0 1618 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 15 0 1 0 84 0 586 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 13 0 1 0 86 0 537 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 14 0 1 0 85 0 600 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 13 0 1 0 86 0 553 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 13 0 1 0 85 0 1631 0.00 0 0.00 38.40 5 0.19 0.00 0 0.00 14 0 1 0 85 0 665 0.00 0 0.00 32.00 4 0.12 0.00 0 0.00 14 0 1 0 85 0 737 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 15 0 1 0 84 0 485 32.00 2 0.06 127.05 770 95.56 0.00 0 0.00 13 0 1 0 85 0 624 0.00 0 0.00 126.95 61 7.55 0.00 0 0.00 13 0 1 0 85 0 1586 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 14 0 1 0 85 0 565 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 14 0 1 0 85 0 488 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 515 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 13 0 1 0 85 0 532 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 14 0 1 0 85 0 1554 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 532 0.00 0 0.00 42.67 3 0.12 0.00 0 0.00 14 0 1 0 85 0 529 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 1 85 0 472 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 0 0 86 0 534 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 14 0 1 0 84 0 1628 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 13 0 1 0 86 0 892 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 13 0 1 0 86 0 604 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 650 0.00 0 0.00 32.00 4 0.12 0.00 0 0.00 13 0 1 0 86 0 568 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 0 1 85 0 1587 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 0 85 0 513 0.00 0 0.00 32.00 5 0.16 0.00 0 0.00 14 0 1 0 85 0 536 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 0 0 85 0 420 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 14 0 0 0 86 0 563 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 14 0 1 0 85 0 1592 0.00 0 0.00 18.67 3 0.05 0.00 0 0.00 13 0 1 0 86 iostat (fast case): tty mfid0 mfid1 cd0 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 505 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 1541 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 1229 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 438 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 326 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 310 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 317 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 1201 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 390 1.00 1 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 357 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 0 0 0 0 100 0 1453 0.00 0 0.00 32.00 65 2.03 0.00 0 0.00 3 0 1 0 95 0 2840 0.00 0 0.00 32.00 291 9.08 0.00 0 0.00 10 0 5 1 85 0 1975 0.00 0 0.00 32.00 307 9.58 0.00 0 0.00 14 0 5 1 79 0 685 0.00 0 0.00 32.00 326 10.18 0.00 0 0.00 13 0 6 1 80 0 691 0.00 0 0.00 32.00 289 9.02 0.00 0 0.00 12 0 6 1 81 0 649 0.00 0 0.00 32.00 268 8.37 0.00 0 0.00 13 0 6 1 80 0 552 0.00 0 0.00 32.00 302 9.43 0.00 0 0.00 14 0 5 1 79 0 1754 0.00 0 0.00 32.00 299 9.33 0.00 0 0.00 14 0 5 2 79 0 630 0.00 0 0.00 32.00 316 9.87 0.00 0 0.00 13 0 6 1 80 0 606 0.00 0 0.00 32.00 287 8.96 0.00 0 0.00 13 0 6 1 80 0 587 0.00 0 0.00 32.00 305 9.52 0.00 0 0.00 15 0 6 1 78 0 576 0.00 0 0.00 32.00 272 8.49 0.00 0 0.00 14 0 6 0 80 0 1763 0.00 0 0.00 32.00 317 9.90 0.00 0 0.00 14 0 6 1 79 0 724 0.00 0 0.00 32.00 379 11.83 0.00 0 0.00 13 0 6 2 79 0 590 4.00 2 0.01 32.00 346 10.80 0.00 0 0.00 14 0 5 1 80 0 580 0.00 0 0.00 32.00 274 8.55 0.00 0 0.00 14 0 6 1 80 0 829 0.00 0 0.00 32.00 338 10.55 0.00 0 0.00 14 0 5 1 80 0 1912 0.00 0 0.00 32.00 326 10.18 0.00 0 0.00 13 0 6 2 79 0 541 0.00 0 0.00 32.00 350 10.93 0.00 0 0.00 13 0 6 1 80 0 652 0.00 0 0.00 32.00 302 9.43 0.00 0 0.00 14 0 6 1 80 0 616 0.00 0 0.00 32.00 297 9.27 0.00 0 0.00 15 0 5 1 79 0 564 0.00 0 0.00 32.00 326 10.18 0.00 0 0.00 14 0 6 1 79 0 1853 0.00 0 0.00 32.00 303 9.46 0.00 0 0.00 13 0 6 1 80 0 752 0.00 0 0.00 32.00 280 8.74 0.00 0 0.00 14 0 6 1 79 0 542 0.00 0 0.00 32.00 300 9.37 0.00 0 0.00 14 0 6 1 79 0 540 32.00 10 0.31 32.00 318 9.93 0.00 0 0.00 14 0 5 1 79 0 688 0.00 0 0.00 32.00 331 10.33 0.00 0 0.00 14 0 5 1 79 0 1863 8.00 2 0.02 32.00 300 9.37 0.00 0 0.00 14 0 6 1 79 0 720 0.00 0 0.00 32.00 306 9.55 0.00 0 0.00 14 0 5 1 80 0 565 0.00 0 0.00 32.00 299 9.33 0.00 0 0.00 14 0 6 0 79 0 1470 0.00 0 0.00 116.51 3413 388.27 0.00 0 0.00 10 0 5 1 83 0 553 0.00 0 0.00 32.00 303 9.46 0.00 0 0.00 13 0 7 1 79 0 1730 0.00 0 0.00 32.00 337 10.52 0.00 0 0.00 14 0 6 0 80 0 649 0.00 0 0.00 32.00 342 10.68 0.00 0 0.00 14 0 6 1 79 0 901 0.00 0 0.00 32.00 317 9.90 0.00 0 0.00 14 0 6 1 80 0 1654 0.00 0 0.00 32.00 328 10.24 0.00 0 0.00 15 0 5 1 79 ktrace (fast case): kdump -f ktrace_noappend.out | grep CALL 2005 mysqld CALL _umtx_op(0x801243140,0xf,0,0,0x7ffffe1f0e20) 2005 mysqld CALL _umtx_op(0x8012431d0,0xf,0,0,0x7ffffd5eade0) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x8012431b8,0xf,0,0,0x7ffffd7ebe40) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243140,0xf,0,0,0x7ffffe1f0e20) 2005 mysqld CALL _umtx_op(0x8012431d0,0xf,0,0,0x7ffffd5eade0) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL recvfrom(0x1f,0xa12389800,0x4,0,0,0) 2005 mysqld CALL recvfrom(0x1f,0xa12389800,0xf1,0,0,0) 2005 mysqld CALL lseek(0xbb,0x12c4fb1c,SEEK_SET) 2005 mysqld CALL read(0xbb,0x7ffffd2e4150,0x14) 2005 mysqld CALL pwrite(0xbb,0xa17a2d014,0x90,0x12c4fb1c) 2005 mysqld CALL pwrite(0xba,0x8a3e91c00,0x400,0x3ee8c00) 2005 mysqld CALL pwrite(0xba,0x8a3e84000,0x400,0x26b6e400) 2005 mysqld CALL pwrite(0xba,0x8a3e84400,0x400,0x26b6e800) 2005 mysqld CALL pwrite(0xba,0x8a3e84800,0x400,0x26b6ec00) 2005 mysqld CALL pwrite(0xba,0x8a3e84c00,0x400,0x26b6f000) 2005 mysqld CALL pwrite(0xba,0x8a93c9800,0x400,0x27a27000) 2005 mysqld CALL pwrite(0xba,0x7ffffd2e33a0,0xb4,0) 2005 mysqld CALL sendto(0x1f,0xa12389800,0xb,0,0,0) 2005 mysqld CALL recvfrom(0x1f,0xa12389800,0x4,0,0,0) 2005 mysqld CALL poll(0x7ffffd2e5d70,0x1,0x1b77400) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243140,0xf,0,0,0x7ffffe1f0e20) 2005 mysqld CALL _umtx_op(0x8012431d0,0xf,0,0,0x7ffffd5eade0) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) ktrace (slow case): kdump -f ktrace_append.out | grep CALL 2005 mysqld CALL _umtx_op(0x8012431b8,0xf,0,0,0x7ffffd7ebe40) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243140,0xf,0,0,0x7ffffe1f0e20) 2005 mysqld CALL _umtx_op(0x8012431d0,0xf,0,0,0x7ffffd5eade0) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243140,0xf,0,0,0x7ffffe1f0e20) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x8012431d0,0xf,0,0,0x7ffffd5eade0) 2005 mysqld CALL recvfrom(0x1f,0xa12389800,0x4,0,0,0) 2005 mysqld CALL recvfrom(0x1f,0xa12389800,0xf1,0,0,0) 2005 mysqld CALL pwrite(0xbb,0xa17a2d015,0x84,0x50f37138) 2005 mysqld CALL pwrite(0xba,0x8a93c9800,0x400,0x27a27000) 2005 mysqld CALL pwrite(0xba,0x8aaf71400,0x400,0x295cec00) 2005 mysqld CALL pwrite(0xba,0x8aaf71800,0x400,0x295cf000) 2005 mysqld CALL pwrite(0xba,0x8aaf71c00,0x400,0x295cf400) 2005 mysqld CALL pwrite(0xba,0x8aaf72000,0x400,0x295cf800) 2005 mysqld CALL pwrite(0xba,0x8aaf7f800,0x400,0x295dd000) 2005 mysqld CALL pwrite(0xba,0x7ffffd2e33a0,0xb4,0) 2005 mysqld CALL sendto(0x1f,0xa12389800,0xb,0,0,0) 2005 mysqld CALL recvfrom(0x1f,0xa12389800,0x4,0,0,0) 2005 mysqld CALL poll(0x7ffffd2e5d70,0x1,0x1b77400) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x8012431b8,0xf,0,0,0x7ffffd7ebe40) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) 2005 mysqld CALL _umtx_op(0x801243110,0xf,0,0,0x7ffffe5f2e20) 2005 mysqld CALL select(0,0,0,0,0x7ffffdfeff70) 2005 mysqld CALL select(0,0,0,0,0x7ffffdbedf70) 2005 mysqld CALL _umtx_op(0x801243128,0xf,0,0,0x7ffffe3f1e10) mfiutil: aegis10:/usr/ports/databases/mysql56-server # mfiutil show adapter mfi0 Adapter: Product Name: LSI MegaRAID SAS 9260-8i Serial Number: SV12003271 Firmware: 12.7.0-0007 RAID Levels: JBOD, RAID0, RAID1, RAID5, RAID6, RAID10, RAID50 Battery Backup: not present NVRAM: 32K Onboard Memory: 512M Minimum Stripe: 8k Maximum Stripe: 1M aegis10:/usr/ports/databases/mysql56-server # mfiutil show config mfi0 Configuration: 2 arrays, 2 volumes, 0 spares array 0 of 2 drives: drive 10 ( 279G) ONLINE SATA drive 11 ( 279G) ONLINE SATA array 1 of 6 drives: drive 9 ( 279G) ONLINE SATA drive 8 ( 279G) ONLINE SATA drive 14 ( 279G) ONLINE SATA drive 12 ( 279G) ONLINE SATA drive 15 ( 279G) ONLINE SATA drive 13 ( 279G) ONLINE SATA volume mfid0 (279G) RAID-1 64k OPTIMAL spans: array 0 volume mfid1 (1115G) RAID-6 64k OPTIMAL spans: array 1 dmesg: Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.1-STABLE #0: Wed Feb 27 20:12:45 UTC 2013 root@aegis10:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Xeon(R) CPU E5606 @ 2.13GHz (2133.38-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206c2 Family = 0x6 Model = 0x2c Stepping = 2 Features=0xbfebfbff Features2=0x29ee3ff AMD Features=0x2c100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 25769803776 (24576 MB) avail memory = 24793403392 (23644 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <052711 APIC2011> FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 18 cpu3 (AP): APIC ID: 20 cpu4 (AP): APIC ID: 32 cpu5 (AP): APIC ID: 34 cpu6 (AP): APIC ID: 50 cpu7 (AP): APIC ID: 52 ioapic0: Changing APIC ID to 1 ioapic1: Changing APIC ID to 3 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, bff00000 (3) failed cpu0: on acpi0 ACPI Warning: Incorrect checksum in table [OEMB] - 0x7D, should be 0x7A (20110527/tbutils-282) cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 450 Event timer "HPET1" frequency 14318180 Hz quality 440 Event timer "HPET2" frequency 14318180 Hz quality 440 Event timer "HPET3" frequency 14318180 Hz quality 440 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci5: on pcib1 igb0: port 0xef80-0xef9f mem 0xfafe0000-0xfaffffff,0xfafc0000-0xfafdffff,0xfaf9c000-0xfaf9ffff irq 28 at device 0.0 on pci5 igb0: Using MSIX interrupts with 9 vectors igb0: Ethernet address: 00:25:90:2b:8e:d4 igb0: Bound queue 0 to cpu 0 igb0: Bound queue 1 to cpu 1 igb0: Bound queue 2 to cpu 2 igb0: Bound queue 3 to cpu 3 igb0: Bound queue 4 to cpu 4 igb0: Bound queue 5 to cpu 5 igb0: Bound queue 6 to cpu 6 igb0: Bound queue 7 to cpu 7 igb1: port 0xef40-0xef5f mem 0xfaf60000-0xfaf7ffff,0xfaf40000-0xfaf5ffff,0xfaf1c000-0xfaf1ffff irq 40 at device 0.1 on pci5 igb1: Using MSIX interrupts with 9 vectors igb1: Ethernet address: 00:25:90:2b:8e:d5 igb1: Bound queue 0 to cpu 0 igb1: Bound queue 1 to cpu 1 igb1: Bound queue 2 to cpu 2 igb1: Bound queue 3 to cpu 3 igb1: Bound queue 4 to cpu 4 igb1: Bound queue 5 to cpu 5 igb1: Bound queue 6 to cpu 6 igb1: Bound queue 7 to cpu 7 pcib2: at device 3.0 on pci0 pci4: on pcib2 mfi0: port 0xd000-0xd0ff mem 0xfae7c000-0xfae7ffff,0xfaec0000-0xfaefffff irq 24 at device 0.0 on pci4 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 pcib3: at device 5.0 on pci0 pci3: on pcib3 pcib4: at device 7.0 on pci0 pci2: on pcib4 pci0: at device 20.0 (no driver attached) pci0: at device 20.1 (no driver attached) pci0: at device 20.2 (no driver attached) pci0: at device 20.3 (no driver attached) pci0: at device 22.0 (no driver attached) pci0: at device 22.1 (no driver attached) pci0: at device 22.2 (no driver attached) pci0: at device 22.3 (no driver attached) pci0: at device 22.4 (no driver attached) pci0: at device 22.5 (no driver attached) pci0: at device 22.6 (no driver attached) pci0: at device 22.7 (no driver attached) uhci0: port 0xcea0-0xcebf irq 16 at device 26.0 on pci0 uhci0: LegSup = 0x2f00 usbus0 on uhci0 uhci1: port 0xce80-0xce9f irq 21 at device 26.1 on pci0 uhci1: LegSup = 0x2f00 usbus1 on uhci1 uhci2: port 0xce20-0xce3f irq 19 at device 26.2 on pci0 uhci2: LegSup = 0x2f00 usbus2 on uhci2 ehci0: mem 0xfadf4000-0xfadf43ff irq 18 at device 26.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 uhci3: port 0xcf20-0xcf3f irq 23 at device 29.0 on pci0 uhci3: LegSup = 0x2f00 usbus4 on uhci3 uhci4: port 0xcf00-0xcf1f irq 19 at device 29.1 on pci0 uhci4: LegSup = 0x2f00 usbus5 on uhci4 uhci5: port 0xcec0-0xcedf irq 18 at device 29.2 on pci0 uhci5: LegSup = 0x2f00 usbus6 on uhci5 ehci1: mem 0xfadf6000-0xfadf63ff irq 23 at device 29.7 on pci0 usbus7: EHCI version 1.0 usbus7 on ehci1 pcib5: at device 30.0 on pci0 pci1: on pcib5 vgapci0: mem 0xf8000000-0xf8ffffff,0xf9ffc000-0xf9ffffff,0xfa000000-0xfa7fffff irq 16 at device 3.0 on pci1 isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0xcff0-0xcff7,0xcfac-0xcfaf,0xcfe0-0xcfe7,0xcfa8-0xcfab,0xcf80-0xcf9f mem 0xfadfa000-0xfadfa7ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart2: <16550 or compatible> port 0x3e8-0x3ef irq 5 on acpi0 qpi0: on motherboard pcib6: pcibus 255 on qpi0 pci255: on pcib6 pcib7: pcibus 254 on qpi0 pci254: on pcib7 orm0: at iomem 0xc0000-0xc7fff,0xd1000-0xd1fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range ctl: CAM Target Layer loaded est0: on cpu0 p4tcc0: on cpu0 est1: on cpu1 p4tcc1: on cpu1 est2: on cpu2 p4tcc2: on cpu2 est3: on cpu3 p4tcc3: on cpu3 est4: on cpu4 p4tcc4: on cpu4 est5: on cpu5 p4tcc5: on cpu5 est6: on cpu6 p4tcc6: on cpu6 est7: on cpu7 p4tcc7: on cpu7 Timecounters tick every 1.000 msec mfi0: 891 (415298016s/0x0020/info) - Shutdown command received from host mfi0: 892 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0079/1000/9261/1000) mfi0: 893 (boot + 3s/0x0020/info) - Firmware version 2.70.03-0862 mfi0: 894 (boot + 5s/0x0020/info) - Board Revision 60A mfi0: 895 (boot + 22s/0x0002/info) - Inserted: PD 08(e0xff/s8) usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 usbus4: 12Mbps Full Speed USB v1.0 usbus5: 12Mbps Full Speed USB v1.0 usbus6: 12Mbps Full Speed USB v1.0 usbus7: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 ugen4.1: at usbus4 uhub4: on usbus4 ugen5.1: at usbus5 uhub5: on usbus5 ugen6.1: at usbus6 uhub6: on usbus6 ugen7.1: at usbus7 uhub7: on usbus7 mfid0 on mfi0 mfid0: 285632MB (584974336 sectors) RAID volume (no label) is optimal mfid1 on mfi0 mfid1: 1142528MB (2339897344 sectors) RAID volume (no label) is optimal mfi0: 896 (boot + 22s/0x0002/info) - Inserted: PD 08(e0xff/s8) Info: enclPd=ffff, scsiType=0, portMap=00, sasAddr=4433221100000000,0000000000000000 mfi0: 897 (boot + 22s/0x0002/info) - Inserted: PD 09(e0xff/s9) mfi0: 898 (boot + 22s/0x0002/info) - Inserted: PD 09(e0xff/s9) Info: enclPd=ffff, scsiType=0, portMap=06, sasAddr=4433221101000000,0000000000000000 mfi0: 899 (boot + 22s/0x0002/info) - Inserted: PD 0a(e0xff/s10) mfi0: 900 (boot + 22s/0x0002/info) - Inserted: PD 0a(e0xff/s10) Info: enclPd=ffff, scsiType=0, portMap=05, sasAddr=4433221103000000,0000000000000000 mfi0: 901 (boot + 22s/0x0002/info) - Inserted: PD 0b(e0xff/s11) mfi0: 902 (boot + 22s/0x0002/info) - Inserted: PD 0b(e0xff/s11) Info: enclPd=ffff, scsiType=0, portMap=03, sasAddr=4433221102000000,0000000000000000 mfi0: 903 (boot + 22s/0x0002/info) - Inserted: PD 0c(e0xff/s12) mfi0: 904 (boot + 22s/0x0002/info) - Inserted: PD 0c(e0xff/s12) Info: enclPd=ffff, scsiType=0, portMap=04, sasAddr=4433221105000000,0000000000000000 mfi0: 905 (boot + 22s/0x0002/info) - Inserted: PD 0d(e0xff/s13) mfi0: 906 (boot + 22s/0x0002/info) - Inserted: PD 0d(e0xff/s13) Info: enclPd=ffff, scsiType=0, portMap=07, sasAddr=4433221107000000,0000000000000000 mfi0: 907 (boot + 22s/0x0002/info) - Inserted: PD 0e(e0xff/s14) mfi0: 908 (boot + 22s/0x0002/info) - Inserted: PD 0e(e0xff/s14) Info: enclPd=ffff, scsiType=0, portMap=02, sasAddr=4433221104000000,0000000000000000 mfi0: 909 (boot + 22s/0x0002/info) - Inserted: PD 0f(e0xff/s15) mfi0: 910 (boot + 22s/0x0002/info) - Inserted: PD 0f(e0xff/s15) Info: enclPd=ffff, scsiType=0, portMap=01, sasAddr=4433221106000000,0000000000000000 mfi0: 911 (415311926s/0x0020/info) - Time established as 02/27/13 20:25:26; (30 seconds since power on) SMP: AP CPU #1 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #2 Launched! cd0 at ahcich0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes) SMP: AP CPU #4 Launched! cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed SMP: AP CPU #3 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #5 Launched! Timecounter "TSC" frequency 2133382316 Hz quality 1000 Root mount waiting for: usbus7 usbus6 usbus5 usbus4 usbus3 usbus2 usbus1 usbus0 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub6: 2 ports with 2 removable, self powered Root mount waiting for: usbus7 usbus3 Root mount waiting for: usbus7 usbus3 uhub3: 6 ports with 6 removable, self powered uhub7: 6 ports with 6 removable, self powered Trying to mount root from ufs:/dev/mfid0p2 [rw]... ugen2.2: at usbus2 ukbd0: on usbus2 kbd2 at ukbd0 ums0: on usbus2 igb0: link state changed to UP From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 4 18:24:16 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B96D78F8 for ; Mon, 4 Mar 2013 18:24:16 +0000 (UTC) (envelope-from garym@oedata.com) Received: from mail-qe0-f54.google.com (mail-qe0-f54.google.com [209.85.128.54]) by mx1.freebsd.org (Postfix) with ESMTP id 82570D2D for ; Mon, 4 Mar 2013 18:24:16 +0000 (UTC) Received: by mail-qe0-f54.google.com with SMTP id i11so3260700qej.27 for ; Mon, 04 Mar 2013 10:24:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type:x-gm-message-state; bh=5647zb5WjSUmNJd95Cl0lRWWZV+b75sdx5Awfo3Q/9I=; b=bHrfDhiPGS/jtvIVgZ6vxEd4IOMPjXFTSTj2K0Ka8RONrxPc81K7vw5jhySPgxoOrx V8wAPWvJTXNujWnVJVb7f6u6Cbo8Hu4Q9M43qMbYXQZ4zERatUVCDRj1cQ9wmKWBeQBe v9DjDCrL9aAQk3BsASVGZdWgQ48UGVv5oWpxJ5UIqrYZiJa3qtTi4+G1WemBmZrVvpGo VeFODwjs/cimijBv2/euDikNI6mub2lBlBQ4nEpWWaYr5ViUdhrfVAj6JtzvUVnc3Rww S+U0MhJH/kgUY6MvLz3zrKpBKsfAd24CEFrm+1JJalJvBZl98ET7FSYQgDnT9mUsUDKU rhjg== MIME-Version: 1.0 X-Received: by 10.224.106.201 with SMTP id y9mr35282695qao.3.1362421448994; Mon, 04 Mar 2013 10:24:08 -0800 (PST) Received: by 10.224.219.137 with HTTP; Mon, 4 Mar 2013 10:24:08 -0800 (PST) In-Reply-To: References: Date: Mon, 4 Mar 2013 11:24:08 -0700 Message-ID: Subject: Re: Need advice on sys5 shm and zero copy sockets From: gary mazzaferro To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQkJ8YW0vFzrHaDQ+23UR6RLn/1oUZvvu9bf+zdxY2Me14SeY8qSAzozM3kIdmJ+nojBY6Gp X-Mailman-Approved-At: Mon, 04 Mar 2013 21:18:35 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 18:24:16 -0000 Hi, Thanks for all the help.. Looks like I'll move forward with recommending linux as a base for our new cloud execution containers. Personally, I thought freebsd would be a technically superior and longer term solution for scientific grid and cloud, but if I can't get support on best architectural practices. I'll need to move to something I that will be supported during eval, design and prototyping processes. -g On Thu, Feb 7, 2013 at 1:22 PM, gary mazzaferro wrote: > Hi, > > I was told to post this question here (Ken Merry), it would be a good > place to get some help. I'm not sure this is doable without a kernel > module, which I don't want to add. > > I'll explain what I'm attempting.. > > I'm designing a high speed rest motor for cloud execution environment. > > 1) I'd like to eliminate copy from the tcp stack to the application(s). > > 2) I'm also sharing the buffers across processes and jails. So I'd > like to preserve the zero-copy in a msg pipe/unix socket > > 3) Some buffers will go to disk file systems. > > > Wish list: > 4) I'd like it to work with sctp because I like it for local networking :) > > 5) I'd like to provision memory pools on a per > application/connection/ip port basis. > > Ultimate Goal: > 6) Additionally, I'm injecting "code" from a foreign process into the > workflow of another process (state machine). The connection between > them will be a signal and shared state information. > > I'm assuming item (6) is a separate issue, but it may impact the direction.. > > I've tried shm with zero copy sockets with linux and it just will not work !! > > BTW, I'm returning to freebsd after far too many years > > cheers, > gary From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 5 12:03:05 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 98AA7687; Tue, 5 Mar 2013 12:03:05 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 9CB929BF; Tue, 5 Mar 2013 12:03:04 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA21556; Tue, 05 Mar 2013 14:02:57 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <5135DEF0.2060802@FreeBSD.org> Date: Tue, 05 Mar 2013 14:02:56 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: Dimitry Andric Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <51323C62.4040506@FreeBSD.org> <51325FB3.7080300@FreeBSD.org> <51337018.30000@FreeBSD.org> <51337C27.7000105@FreeBSD.org> <5133AC3B.5040504@FreeBSD.org> <51348A6A.3070606@FreeBSD.org> In-Reply-To: <51348A6A.3070606@FreeBSD.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 12:03:05 -0000 on 04/03/2013 13:50 Dimitry Andric said the following: > Actually, that way of fixing breaks it for gcc, so it should really be > fixed in sort_iidescs() instead. See attached diff; I verified it works > for both gcc-compiled and clang-compiled objects. Impossible! It looks like the attached diff was a nop change :-) BTW, so changing clang to match gcc is not an option? I do not insist, just curious. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 5 14:57:35 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2C80371C; Tue, 5 Mar 2013 14:57:35 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:7b8:3a7:1:2d0:b7ff:fea0:8c26]) by mx1.freebsd.org (Postfix) with ESMTP id DADE0681; Tue, 5 Mar 2013 14:57:34 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a] (spaceball.andric.com [IPv6:2001:7b8:3a7:0:204:4bff:fe01:de8a]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 159AE5C43; Tue, 5 Mar 2013 15:57:32 +0100 (CET) Message-ID: <513607DA.60901@FreeBSD.org> Date: Tue, 05 Mar 2013 15:57:30 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20130117 Thunderbird/19.0 MIME-Version: 1.0 To: Andriy Gapon Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <51323C62.4040506@FreeBSD.org> <51325FB3.7080300@FreeBSD.org> <51337018.30000@FreeBSD.org> <51337C27.7000105@FreeBSD.org> <5133AC3B.5040504@FreeBSD.org> <51348A6A.3070606@FreeBSD.org> <5135DEF0.2060802@FreeBSD.org> In-Reply-To: <5135DEF0.2060802@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 14:57:35 -0000 On 2013-03-05 13:02, Andriy Gapon wrote: > on 04/03/2013 13:50 Dimitry Andric said the following: >> Actually, that way of fixing breaks it for gcc, so it should really be >> fixed in sort_iidescs() instead. See attached diff; I verified it works >> for both gcc-compiled and clang-compiled objects. > > Impossible! It looks like the attached diff was a nop change :-) > > BTW, so changing clang to match gcc is not an option? > I do not insist, just curious. I could not find a good argument to present to upstream to change this behaviour, except "gcc has been doing it for x years, without a good reason". :-) And I did not want to make yet another FreeBSD specific change. In my opinion, gcc is losing information here, e.g. the original pathname to the source file. There is no reason debugging tools cannot be fixed to be able to cope with this. From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 5 16:04:03 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CF0E0990 for ; Tue, 5 Mar 2013 16:04:03 +0000 (UTC) (envelope-from erdgeist@erdgeist.org) Received: from elektropost.org (elektropost.org [217.13.206.130]) by mx1.freebsd.org (Postfix) with ESMTP id 22802AA6 for ; Tue, 5 Mar 2013 16:04:02 +0000 (UTC) Received: (qmail 7757 invoked from network); 5 Mar 2013 16:03:55 -0000 Received: from elektropost.org (HELO elektropost.org) (erdgeist@erdgeist.org) by elektropost.org with AES256-SHA encrypted SMTP; 5 Mar 2013 16:03:55 -0000 Date: Tue, 5 Mar 2013 17:03:31 +0100 (CET) From: Dirk Engling To: freebsd-hackers@freebsd.org Subject: pop filters from kqueue Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 16:04:03 -0000 Dear fellow FreeBSD hackers, while writing a daemon that uses a kqueue to keep track of forked processes and pipes to userland client code etc, I noticed a lack of features to implement a proper shutdown without holding data redundantly. When my daemon quits, I can not ask the kqueue for my installed filters and get back the udata I passed to the kqueue. This is unfortunate, because I like the idea of having only one owner per memory allocation. The most obvious use would be a per-fd-state held in a memory block. When passing it to kevent() via the udata entry, I would make this filter the owner of my allocation. However, when gracefully shutting down, my daemon has no way of retrieving all the values passed to the filters. For most cases that may be okay: memory allocations will just be thrown away on exit(), anyway. But once I need to clean up external state, like sending signals to processes I installed an EVFILT_PROC for etc, I need to keep a redundant list of pids and the associated udata. This violates the rule of strict ownership and introduces room for inconsistencies. Is there a specific reason I have overlooked that would forbid popping untriggered filters from my kqueue? Or is there even a way to do so that I have missed? Regards, erdgeist From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 5 16:53:11 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3E12E35F for ; Tue, 5 Mar 2013 16:53:11 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 29FFCF01 for ; Tue, 5 Mar 2013 16:53:10 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id EA7F11A3C22; Tue, 5 Mar 2013 08:53:08 -0800 (PST) Message-ID: <513622F2.20902@mu.org> Date: Tue, 05 Mar 2013 08:53:06 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130216 Thunderbird/17.0.3 MIME-Version: 1.0 To: Dirk Engling Subject: Re: pop filters from kqueue References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 16:53:11 -0000 On 3/5/13 8:03 AM, Dirk Engling wrote: > Dear fellow FreeBSD hackers, > > while writing a daemon that uses a kqueue to keep track of forked > processes and pipes to userland client code etc, I noticed a lack of > features to implement a proper shutdown without holding data redundantly. > > When my daemon quits, I can not ask the kqueue for my installed > filters and get back the udata I passed to the kqueue. > > This is unfortunate, because I like the idea of having only one owner > per memory allocation. The most obvious use would be a per-fd-state > held in a memory block. When passing it to kevent() via the udata > entry, I would make this filter the owner of my allocation. > > However, when gracefully shutting down, my daemon has no way of > retrieving all the values passed to the filters. For most cases that > may be okay: > memory allocations will just be thrown away on exit(), anyway. > > But once I need to clean up external state, like sending signals to > processes I installed an EVFILT_PROC for etc, I need to keep a > redundant list of pids and the associated udata. This violates the > rule of strict ownership and introduces room for inconsistencies. > > Is there a specific reason I have overlooked that would forbid popping > untriggered filters from my kqueue? Or is there even a way to do so > that I have missed? > I'm not sure if kqueue support this, however adding such a facility might be OK. Another way to handle this is just to make your udata pointers all point to items in a doubly linked list with the head structure aware of which kqueue. Then at the end you can just traverse the list of items you have not yet popped off. The only pain here is that it requires managing a doubly linked list and additional pointer dereferences. -Alfred From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 5 17:27:42 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DF4CED34 for ; Tue, 5 Mar 2013 17:27:42 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by mx1.freebsd.org (Postfix) with ESMTP id 73A35149 for ; Tue, 5 Mar 2013 17:27:42 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1UCvQ3-00070i-Gs for freebsd-hackers@freebsd.org; Tue, 05 Mar 2013 18:12:51 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 05 Mar 2013 18:12:51 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 05 Mar 2013 18:12:51 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Subject: Re: MySQL Data_free = 0 Slowness (appending to files slow?) Date: Tue, 05 Mar 2013 18:12:19 +0100 Lines: 117 Message-ID: References: <20130304155020.77e470da@pleiades.nextvenue.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig90C1B9441DFB393EA253B2B0" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120812 Thunderbird/14.0 In-Reply-To: <20130304155020.77e470da@pleiades.nextvenue.com> X-Enigmail-Version: 1.4.3 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 17:27:42 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig90C1B9441DFB393EA253B2B0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 04/03/2013 21:50, Nick Evans wrote: > iostat (slow case): >=20 > tty mfid0 mfid1 cd0 = cpu > tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy = in id > 0 479 0.00 0 0.00 80.00 2 0.16 0.00 0 0.00 11 0 1 = 0 88 > 0 600 0.00 0 0.00 32.00 3 0.09 0.00 0 0.00 13 0 1 = 0 86 > 0 553 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 13 0 1 = 0 85 > 0 1631 0.00 0 0.00 38.40 5 0.19 0.00 0 0.00 14 0 1 = 0 85 > 0 665 0.00 0 0.00 32.00 4 0.12 0.00 0 0.00 14 0 1 = 0 85 > 0 737 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 15 0 1 = 0 84 > 0 485 32.00 2 0.06 127.05 770 95.56 0.00 0 0.00 13 0 1= 0 85 > 0 624 0.00 0 0.00 126.95 61 7.55 0.00 0 0.00 13 0 1= 0 85 > 0 1586 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 14 0 1 = 0 85 > 0 565 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 14 0 1 = 0 85 > 0 488 0.00 0 0.00 32.00 2 0.06 0.00 0 0.00 14 0 1 = 0 85 > 0 515 0.00 0 0.00 32.00 1 0.03 0.00 0 0.00 13 0 1 = 0 85 > iostat (fast case): >=20 > tty mfid0 mfid1 cd0 = cpu > tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy = in id > 0 542 0.00 0 0.00 32.00 300 9.37 0.00 0 0.00 14 0 6 = 1 79 > 0 540 32.00 10 0.31 32.00 318 9.93 0.00 0 0.00 14 0 5 = 1 79 > 0 688 0.00 0 0.00 32.00 331 10.33 0.00 0 0.00 14 0 5 = 1 79 > 0 1863 8.00 2 0.02 32.00 300 9.37 0.00 0 0.00 14 0 6 = 1 79 > 0 720 0.00 0 0.00 32.00 306 9.55 0.00 0 0.00 14 0 5 = 1 80 > 0 565 0.00 0 0.00 32.00 299 9.33 0.00 0 0.00 14 0 6 = 0 79 > 0 1470 0.00 0 0.00 116.51 3413 388.27 0.00 0 0.00 10 0 = 5 1 83 > 0 553 0.00 0 0.00 32.00 303 9.46 0.00 0 0.00 13 0 7 = 1 79 > 0 1730 0.00 0 0.00 32.00 337 10.52 0.00 0 0.00 14 0 6 = 0 80 > 0 649 0.00 0 0.00 32.00 342 10.68 0.00 0 0.00 14 0 6 = 1 79 > 0 901 0.00 0 0.00 32.00 317 9.90 0.00 0 0.00 14 0 6 = 1 80 > 0 1654 0.00 0 0.00 32.00 328 10.24 0.00 0 0.00 15 0 5 = 1 79 I don't know if you've noticed, but: 1) In your slow case, the load is almost entirely in userspace (USR), i.e. mysql, while in the fast case, the kernel load (SYS) is 5x-6x greater. Unless the first case actually excercises your disks more (unlikely since you have almost no IO load at all) it means that it's mysql that is slow, not the kernel. 2) Ordinary iostat doesn't show the disk busy-time estimation. You need to use "-x" (e.g. "iostat -x 1") to see the service time and busy percentage stats. 3) you seem to have bursts of traffic, probably writing, which may be caused by the file system IFF mysql is not doing anything to fsync the data sooner (i.e. you have everything in a single transaction), but the burst in the "fast" case is 388 MB while the burst in the "slow" case is 100 MB 4) your benchmark seems to be single-threaded 5) You have a lot of 32 KB transactions, which probably means you have created the file system in 9.x with the default options, leading to 32 KB blocks. This is probably not the source of your current problems, but in the future you might want to match the block size to innodb's block size (16 KiB). You should also do that on the RAID array level and match the block alignments. 6) Are you using innodb_file_per_table? You probably should. --------------enig90C1B9441DFB393EA253B2B0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlE2J3QACgkQ/QjVBj3/HSwF6ACgmOuvaaIfoX44ZWWvfyLbUugd qVEAn1CmbceGx7OfDdLpZfNGFmd9G9+D =D2ru -----END PGP SIGNATURE----- --------------enig90C1B9441DFB393EA253B2B0-- From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 5 18:35:50 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8ECF1130 for ; Tue, 5 Mar 2013 18:35:50 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 014336D5 for ; Tue, 5 Mar 2013 18:35:49 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r25IZYNa003317 for ; Tue, 5 Mar 2013 19:35:34 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r25IZYRo003314 for ; Tue, 5 Mar 2013 19:35:34 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Tue, 5 Mar 2013 19:35:34 +0100 (CET) From: Wojciech Puchar To: freebsd-hackers@freebsd.org Subject: gjournal +UFS - anyone actually use it? Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Tue, 05 Mar 2013 19:35:34 +0100 (CET) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 18:35:50 -0000 if yes - how about performance on production server. i think about journal on SSD. From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 5 19:21:00 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 696A1A28 for ; Tue, 5 Mar 2013 19:21:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4833A87B for ; Tue, 5 Mar 2013 19:21:00 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A0B21B992; Tue, 5 Mar 2013 14:20:59 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: Need advice on sys5 shm and zero copy sockets Date: Tue, 5 Mar 2013 14:11:24 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201303051411.24608.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 05 Mar 2013 14:20:59 -0500 (EST) Cc: gary mazzaferro X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 19:21:00 -0000 On Monday, March 04, 2013 1:24:08 pm gary mazzaferro wrote: > Hi, > > Thanks for all the help.. Looks like I'll move forward with > recommending linux as a base for our new cloud execution containers. > > Personally, I thought freebsd would be a technically superior and > longer term solution for scientific grid and cloud, but if I can't get > support on best architectural practices. I'll need to move to > something I that will be supported during eval, design and prototyping > processes. There is not anything in stock FreeBSD that currently does zero-copy sockets for TCP. You could add such a thing, but you would have to implement your own. :( Some of the building blocks are in place. For example, you can create POSIX shared memory objects via shm_open() (and FreeBSD has an extension where a path of SHM_ANON creates anonymous, unnamed objects) and pass that fd into the kernel where an ioctl handler can map it into KVA using shm_map() and shm_unmap(). You'd have to change TCP to do something useful with this buffer however. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 6 02:04:03 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8035DE1B for ; Wed, 6 Mar 2013 02:04:03 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-qa0-f41.google.com (mail-qa0-f41.google.com [209.85.216.41]) by mx1.freebsd.org (Postfix) with ESMTP id 22BBD9E9 for ; Wed, 6 Mar 2013 02:04:02 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id bs12so2250446qab.7 for ; Tue, 05 Mar 2013 18:03:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:subject:references:from:content-type:x-mailer :in-reply-to:message-id:date:to:content-transfer-encoding :mime-version:x-gm-message-state; bh=e3jAn0BhVnLULKibl8Tw3hg23h408ZfjsDNntidrgcE=; b=Bro0cyQ87B6QuKB5mUevUEpe6SbvHbBSNKxcJeD8yj4POXE+cO1QOffRC2s/cfxnQM o/vTuvPqaB5nmGqCPwnvMAPh7aNccD4IZabYQFalW9v3950iE+Rc6tFUe3oPzRWoTEe+ nJyPidTDph3Ne+AVjFFMrismWb8jKxq1K0giZ6e6GoDmL8Ma3TEuRVMRhi/8v+j9QFW1 IWQRmnUP8GEpJjo3MhTETcbI9qHjPXWIt9OxEp0DVBy5/cUraAUT3CFNA8baIiPKc8RF 7RoZgsseYDlDrGjDazrTTszqIfi11s1Yo4pV1xLBD16mr8ytq43idNvR7HqtJ6iNwY2q aT0Q== X-Received: by 10.229.136.69 with SMTP id q5mr9468788qct.108.1362535436123; Tue, 05 Mar 2013 18:03:56 -0800 (PST) Received: from [192.168.11.105] (ool-4a58bea2.dyn.optonline.net. [74.88.190.162]) by mx.google.com with ESMTPS id t7sm36589306qey.2.2013.03.05.18.03.52 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 05 Mar 2013 18:03:55 -0800 (PST) Subject: Re: gjournal +UFS - anyone actually use it? References: From: Mark Saad Content-Type: text/plain; charset=us-ascii X-Mailer: iPhone Mail (10B146) In-Reply-To: Message-Id: <15BA8CB5-A10A-4FA9-AC5F-1BA2B7720E5E@longcount.org> Date: Tue, 5 Mar 2013 21:03:51 -0500 To: "freebsd-hackers@freebsd.org" Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) X-Gm-Message-State: ALoCoQkxVccIju/1ruDGjdTsj2XClgeOQfY+NIS3bc0d4q2atRNbDMB7UEPIDfSNay5P1YlYIRUL X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 02:04:03 -0000 On Mar 5, 2013, at 1:35 PM, Wojciech Puchar = wrote: > if yes - how about performance on production server. I have used gjournal on 7.x amd64 . Using in disk journals . This was on a h= p dl380 g5 using 6 300gb sas disks in a raid 1+0 .=20 I was using it to store large MySQL myisam tables , speed was acceptable at t= he time . I never had any fs corruption and it worked as expected .=20 At the time I set it up I remember there was some chatter about how slow gjo= urnal was compared to ufs with softupdate .=20 Fast forward to today I almost always use ufs with softupdate journal , new i= n FreeBSD 9.0 and available as a patch to 6.x, 7.x , and 8.x=20 This is better supported now , as more people use it in new 9.x builds . > i think about journal on SSD. I believe this is only and option in geom journal , I am not sure if you can= relocate a suj journal to an alternate disk . It should be faster . All I c= an say here is give it a try and see what happens .=20 Hope that helps=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= --- Mark saad | mark.saad@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 6 10:56:01 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DD53C8F for ; Wed, 6 Mar 2013 10:56:01 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 3A208163 for ; Wed, 6 Mar 2013 10:56:00 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r26AtsYG002299; Wed, 6 Mar 2013 11:55:54 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r26Atr36002296; Wed, 6 Mar 2013 11:55:54 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 6 Mar 2013 11:55:53 +0100 (CET) From: Wojciech Puchar To: Mark Saad Subject: Re: gjournal +UFS - anyone actually use it? In-Reply-To: <15BA8CB5-A10A-4FA9-AC5F-1BA2B7720E5E@longcount.org> Message-ID: References: <15BA8CB5-A10A-4FA9-AC5F-1BA2B7720E5E@longcount.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 06 Mar 2013 11:55:54 +0100 (CET) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 10:56:01 -0000 > I was using it to store large MySQL myisam tables , speed was acceptable at the time . I never had any fs corruption and it worked as expected . > > At the time I set it up I remember there was some chatter about how slow gjournal was compared to ufs with softupdate . did some tests yesterday on 25GB partition. i simulated journal on SSD using 2GB malloc backed ramdisk (md0). UFS partition mounted async as gjournal recommends. test: unpacking kernel sources to multiple directories until disk gets full simulates write heavy I/O. gjournal+SATA drive (seagate constellation ES 500GB) with write cache disabled - 14 minutes 20 seconds. write cache enabled - 14 minutes 5 seconds (nearly no difference). UFS+journalled softupdates, no gjournal, disk write cache on - 26 minutes 44 seconds. disk write cache off - was too lazy to wait after an hour. With gjournal it is not only faster, but it doesn't make other I/O activity crawling. > > Fast forward to today I almost always use ufs with softupdate journal , new in FreeBSD 9.0 and available as a patch to 6.x, 7.x , and 8.x The problem is as follows: SU+J makes sure that metadata will get consistent. NOT DATA. And this is quite a mess if you get UPS failure under high load. gjournal does journal everything. > This is better supported now , as more people use it in new 9.x builds . > >> i think about journal on SSD. > > I believe this is only and option in geom journal , SSDs are not expensive today. i can get 128GB SSD and create 20GB journal just to limit wear. and possibly use the rest of SSD to store read-intensive data. the way gjournal writes to journal device (sequential) is very friendly for SSD too. Small SLC-based SSD would be best but i don't see anything on the market with acceptable price for now. > I am not sure if you can relocate a suj journal to an alternate disk. no you can't. But still you will end with consistent metadata but not consistent data. I recovered it but still it took a time and lots of checking. gjournal doesn't seem to be elegant in case of journal failure (i simulated it with forced removal of ramdisk with mdconfig). TONS of messages in logs, but still - no data loss, just you have to shutdown system, boot from pendrive, remove journal, fsck (just for sure), and then add journal again From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 6 15:38:08 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1F4E6438; Wed, 6 Mar 2013 15:38:08 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id C26A31C4; Wed, 6 Mar 2013 15:38:07 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2A5D1B917; Wed, 6 Mar 2013 10:38:07 -0500 (EST) From: John Baldwin To: Ian Lepore Subject: Re: rtprio_thread trouble Date: Wed, 6 Mar 2013 09:17:13 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: <1361559960.1185.81.camel@revolution.hippie.lan> <201302261529.24862.jhb@freebsd.org> <1362081556.1195.62.camel@revolution.hippie.lan> In-Reply-To: <1362081556.1195.62.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201303060917.13642.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 06 Mar 2013 10:38:07 -0500 (EST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 15:38:08 -0000 On Thursday, February 28, 2013 2:59:16 pm Ian Lepore wrote: > On Tue, 2013-02-26 at 15:29 -0500, John Baldwin wrote: > > On Friday, February 22, 2013 2:06:00 pm Ian Lepore wrote: > > > I ran into some trouble with rtprio_thread() today. I have a worker > > > thread that I want to run at idle priority most of the time, but if it > > > falls too far behind I'd like to bump it back up to regular timeshare > > > priority until it catches up. In a worst case, the system is > > > continuously busy and something scheduled at idle priority is never > > > going to run (for some definition of 'never'). > > > > > > What I found is that in this worst case, even after my main thread has > > > used rtprio_thread() to change the worker thread back to > > > RTP_PRIO_NORMAL, the worker thread never gets scheduled. This is with > > > the 4BSD scheduler but it appears that the same would be the case with > > > ULE, based on code inspection. I find that this fixes it for 4BSD, and > > > I think the same would be true for ULE... > > > > > > --- a/sys/kern/sched_4bsd.c Wed Feb 13 12:54:36 2013 -0700 > > > +++ b/sys/kern/sched_4bsd.c Fri Feb 22 11:55:35 2013 -0700 > > > @@ -881,6 +881,9 @@ sched_user_prio(struct thread *td, u_cha > > > return; > > > oldprio = td->td_user_pri; > > > td->td_user_pri = prio; > > > + if (td->td_flags & TDF_BORROWING && td->td_priority <= prio) > > > + return; > > > + sched_priority(td, prio); > > > } > > > > > > void > > > > > > But I'm not sure if this would have any negative side effects, > > > especially since in the ULE case there's a comment on this function that > > > specifically notes that it changes the the user priority without > > > changing the current priority (but it doesn't say why that matters). > > > > > > Is this a reasonable way to fix this problem, or is there a better way? > > > > This will lose the "priority boost" afforded to interactive threads when they > > sleep in the kernel in the 4BSD scheduler. You aren't supposed to drop the > > user priority to loose this boost until userret(). You could perhaps try > > only altering the priority if the new user pri is lower than your current > > priority (and then you don't have to check TDF_BORROWING I believe): > > > > if (prio < td->td_priority) > > sched_priority(td, prio); > > > > That's just the sort of insight I was looking for, thanks. That made me > look at the code more and think harder about the problem I'm trying to > solve, and I concluded that doing it within the scheduler is all wrong. > > That led me to look elsewhere, and I discovered the change you made in > r228207, which does almost what I want, but your change does it only for > realtime priorities, and I need a similar effect for idle priorities. > What I came up with is a bit different than yours (attached below) and > I'd like your thoughts on it. > > I start with the same test as yours: if sched_user_prio() didn't > actually change the user priority (due to borrowing), do nothing. Then > mine differs: call sched_prio() to effect the change only if either the > old or the new priority class is not timeshare. > > My reasoning for the second half of the test is that if it's a change in > timeshare priority then the scheduler is going to adjust that priority > in a way that completely wipes out the requested change anyway, so > what's the point? (If that's not true, then allowing a thread to change > its own timeshare priority would subvert the scheduler's adjustments and > let a cpu-bound thread monopolize the cpu; if allowed at all, that > should require priveleges.) > > On the other hand, if either the old or new priority class is not > timeshare, then the scheduler doesn't make automatic adjustments, so we > should honor the request and make the priority change right away. The > reason the old class gets caught up in this is the very reason I'm > wanting to make a change: when thread A changes the priority of its > child thread B from idle back to timeshare, thread B never actually gets > moved to a timeshare-range run queue unless there are some idle cycles > available to allow it to first get scheduled again as an idle thread. > > Finally, my change doesn't consider the td == curthread situation at > all, because I don't see how that's germane. This is the thing I'm > least sure of -- I don't at all understand why the old code (even before > your changes) had that test. The old code had that flagged as "XXX > dubious" (a comment a bit too cryptic to be useful). I think your change is correct. One style nit: please sort the order of variables (oldclass comes before oldpri). -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 6 16:10:44 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1CD8BC4D; Wed, 6 Mar 2013 16:10:44 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) by mx1.freebsd.org (Postfix) with ESMTP id CA0133C4; Wed, 6 Mar 2013 16:10:43 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1UDGvT-000LhG-3f; Wed, 06 Mar 2013 16:10:43 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r26GAeRa003230; Wed, 6 Mar 2013 09:10:40 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX181Xm+C3ypxi/qrAWxMTgu+ Subject: Re: rtprio_thread trouble From: Ian Lepore To: John Baldwin In-Reply-To: <201303060917.13642.jhb@freebsd.org> References: <1361559960.1185.81.camel@revolution.hippie.lan> <201302261529.24862.jhb@freebsd.org> <1362081556.1195.62.camel@revolution.hippie.lan> <201303060917.13642.jhb@freebsd.org> Content-Type: text/plain; charset="us-ascii" Date: Wed, 06 Mar 2013 09:10:40 -0700 Message-ID: <1362586240.1291.110.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 16:10:44 -0000 On Wed, 2013-03-06 at 09:17 -0500, John Baldwin wrote: > On Thursday, February 28, 2013 2:59:16 pm Ian Lepore wrote: > > On Tue, 2013-02-26 at 15:29 -0500, John Baldwin wrote: > > > On Friday, February 22, 2013 2:06:00 pm Ian Lepore wrote: > > > > I ran into some trouble with rtprio_thread() today. I have a worker > > > > thread that I want to run at idle priority most of the time, but if it > > > > falls too far behind I'd like to bump it back up to regular timeshare > > > > priority until it catches up. In a worst case, the system is > > > > continuously busy and something scheduled at idle priority is never > > > > going to run (for some definition of 'never'). > > > > > > > > What I found is that in this worst case, even after my main thread has > > > > used rtprio_thread() to change the worker thread back to > > > > RTP_PRIO_NORMAL, the worker thread never gets scheduled. This is with > > > > the 4BSD scheduler but it appears that the same would be the case with > > > > ULE, based on code inspection. I find that this fixes it for 4BSD, and > > > > I think the same would be true for ULE... > > > > > > > > --- a/sys/kern/sched_4bsd.c Wed Feb 13 12:54:36 2013 -0700 > > > > +++ b/sys/kern/sched_4bsd.c Fri Feb 22 11:55:35 2013 -0700 > > > > @@ -881,6 +881,9 @@ sched_user_prio(struct thread *td, u_cha > > > > return; > > > > oldprio = td->td_user_pri; > > > > td->td_user_pri = prio; > > > > + if (td->td_flags & TDF_BORROWING && td->td_priority <= prio) > > > > + return; > > > > + sched_priority(td, prio); > > > > } > > > > > > > > void > > > > > > > > But I'm not sure if this would have any negative side effects, > > > > especially since in the ULE case there's a comment on this function that > > > > specifically notes that it changes the the user priority without > > > > changing the current priority (but it doesn't say why that matters). > > > > > > > > Is this a reasonable way to fix this problem, or is there a better way? > > > > > > This will lose the "priority boost" afforded to interactive threads when they > > > sleep in the kernel in the 4BSD scheduler. You aren't supposed to drop the > > > user priority to loose this boost until userret(). You could perhaps try > > > only altering the priority if the new user pri is lower than your current > > > priority (and then you don't have to check TDF_BORROWING I believe): > > > > > > if (prio < td->td_priority) > > > sched_priority(td, prio); > > > > > > > That's just the sort of insight I was looking for, thanks. That made me > > look at the code more and think harder about the problem I'm trying to > > solve, and I concluded that doing it within the scheduler is all wrong. > > > > That led me to look elsewhere, and I discovered the change you made in > > r228207, which does almost what I want, but your change does it only for > > realtime priorities, and I need a similar effect for idle priorities. > > What I came up with is a bit different than yours (attached below) and > > I'd like your thoughts on it. > > > > I start with the same test as yours: if sched_user_prio() didn't > > actually change the user priority (due to borrowing), do nothing. Then > > mine differs: call sched_prio() to effect the change only if either the > > old or the new priority class is not timeshare. > > > > My reasoning for the second half of the test is that if it's a change in > > timeshare priority then the scheduler is going to adjust that priority > > in a way that completely wipes out the requested change anyway, so > > what's the point? (If that's not true, then allowing a thread to change > > its own timeshare priority would subvert the scheduler's adjustments and > > let a cpu-bound thread monopolize the cpu; if allowed at all, that > > should require priveleges.) > > > > On the other hand, if either the old or new priority class is not > > timeshare, then the scheduler doesn't make automatic adjustments, so we > > should honor the request and make the priority change right away. The > > reason the old class gets caught up in this is the very reason I'm > > wanting to make a change: when thread A changes the priority of its > > child thread B from idle back to timeshare, thread B never actually gets > > moved to a timeshare-range run queue unless there are some idle cycles > > available to allow it to first get scheduled again as an idle thread. > > > > Finally, my change doesn't consider the td == curthread situation at > > all, because I don't see how that's germane. This is the thing I'm > > least sure of -- I don't at all understand why the old code (even before > > your changes) had that test. The old code had that flagged as "XXX > > dubious" (a comment a bit too cryptic to be useful). > > I think your change is correct. One style nit: please sort the order of > variables (oldclass comes before oldpri). > Thanks for the review. I've been running my change on one of our products in an 8.2 environment, and on some arm platforms running -current, and it seems to be working well. Alphabetizing: Grrr, yeah. I had it that way at first, but it just offended my sensibilities to separate related values with an unrelated value. IMO, clumping related things together makes way more sense than ordering things based on a song I learned when I was 3 years old. ::sigh:: I'll put it back the other way. :) -- Ian From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 6 18:30:37 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2C3B75A4; Wed, 6 Mar 2013 18:30:37 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from mailbox.talkpoint.com (mailbox.talkpoint.com [204.141.10.74]) by mx1.freebsd.org (Postfix) with ESMTP id F1E14DE1; Wed, 6 Mar 2013 18:30:36 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailbox.talkpoint.com (Postfix) with ESMTP id 04EFAF66006; Wed, 6 Mar 2013 13:30:30 -0500 (EST) X-Virus-Scanned: amavisd-new at talkpoint.com Received: from mailbox.talkpoint.com ([127.0.0.1]) by localhost (mailbox.talkpoint.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PaZbCIxsonOB; Wed, 6 Mar 2013 13:30:25 -0500 (EST) Received: from pleiades.nextvenue.com (pleiades.nextvenue.com [204.141.15.194]) by mailbox.talkpoint.com (Postfix) with ESMTP id 325A0F66001; Wed, 6 Mar 2013 13:30:25 -0500 (EST) Date: Wed, 6 Mar 2013 13:30:24 -0500 From: Nick Evans To: Ivan Voras Subject: Re: MySQL Data_free = 0 Slowness (appending to files slow?) Message-ID: <20130306133024.24fcacd4@pleiades.nextvenue.com> In-Reply-To: References: <20130304155020.77e470da@pleiades.nextvenue.com> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.6; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 18:30:37 -0000 On Tue, 05 Mar 2013 18:12:19 +0100 Ivan Voras wrote: > I don't know if you've noticed, but: > > 1) In your slow case, the load is almost entirely in userspace (USR), > i.e. mysql, while in the fast case, the kernel load (SYS) is 5x-6x > greater. Unless the first case actually excercises your disks more > (unlikely since you have almost no IO load at all) it means that it's > mysql that is slow, not the kernel. > > 2) Ordinary iostat doesn't show the disk busy-time estimation. You > need to use "-x" (e.g. "iostat -x 1") to see the service time and busy > percentage stats. > > 3) you seem to have bursts of traffic, probably writing, which may be > caused by the file system IFF mysql is not doing anything to fsync the > data sooner (i.e. you have everything in a single transaction), but > the burst in the "fast" case is 388 MB while the burst in the "slow" > case is 100 MB > > 4) your benchmark seems to be single-threaded > > 5) You have a lot of 32 KB transactions, which probably means you have > created the file system in 9.x with the default options, leading to 32 > KB blocks. This is probably not the source of your current problems, > but in the future you might want to match the block size to innodb's > block size (16 KiB). You should also do that on the RAID array level > and match the block alignments. > > 6) Are you using innodb_file_per_table? You probably should. > 1) I had not noticed that. Thanks. 2) Here's iostat -x 1 output for both cases. slow: http://pleiades.nextvenue.com/mysql/iostat_x_slow.txt fast: http://pleiades.nextvenue.com/mysql/iostat_x_fast.txt 3) The load generators are constant, so I'm guessing this is file system related, but I can't see into MySQL very much to tell for sure. Would the ktraces show mysql doing fsync? These tables are all MyISAM so no transactions as such, and each super-smack agent is running insert queries individually. 4) Technically 2 threads. I am only running a single super-smack on each load server of which there are two. I tried to keep the test as simple as possible to start. 5) I'll trash the array and change the stripe to 16k as well as the block size and see if it helps. 6) As I said before, these are MyISAM tables, so they are already separate table files. Unfortunately, MySQL doesn't seem to have a reserve x space per table for MyISAM tables. I can manually do this for my high access tables by inserting bogus rows and then deleting them (which I may do) as a temporary fix. From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 7 00:07:49 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 85CBBFDB for ; Thu, 7 Mar 2013 00:07:49 +0000 (UTC) (envelope-from nicholas@nickcwilson.co.uk) Received: from mail-qa0-f41.google.com (mail-qa0-f41.google.com [209.85.216.41]) by mx1.freebsd.org (Postfix) with ESMTP id 29A5531E for ; Thu, 7 Mar 2013 00:07:48 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id bs12so2886867qab.7 for ; Wed, 06 Mar 2013 16:07:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:x-received:x-originating-ip:in-reply-to :references:date:message-id:subject:from:to:content-type :x-gm-message-state; bh=65J64GXFU/gaKMrKgsR7EFOVInlAAtV7ng/vu1ht3Sk=; b=S8hlq8UTCR2UkK8YcLP4IeHwkWKVzZB3NTOnGguRzkp1y0fgJGM0KxKT2syCwv9d/V 6IoZqoiUFGFmz7d8ieNm1XyBFytmjlnz+E56nON8cO7UufT2BjXQYXPxwHbmLKTMOQjP oQ97BhlXmJG1J0RzMUFTHcww3lTQiyFgDMqqsHPHe0P+MqPq3sbzHniT5ZDP4rcAvDp9 xf/Es+5oS4HLIEzwvQEfiQ6XbEGYWpfQyYhym+5gGSw20YbA3JKOtKGg4PsCFBZiCxLM N1gWW43W7PoUI24Cc1J2sii7h2qhx4yHsGgN/CnIVMs7c69UHqrIHT1Sh34yP5noF3fG 5+bg== X-Received: by 10.224.173.147 with SMTP id p19mr47971957qaz.78.1362614867784; Wed, 06 Mar 2013 16:07:47 -0800 (PST) Received: from mail-qe0-f43.google.com (mail-qe0-f43.google.com [209.85.128.43]) by mx.google.com with ESMTPS id c2sm1107897qeg.6.2013.03.06.16.07.46 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 06 Mar 2013 16:07:47 -0800 (PST) Received: by mail-qe0-f43.google.com with SMTP id 1so5616009qee.16 for ; Wed, 06 Mar 2013 16:07:46 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.49.52.70 with SMTP id r6mr52320893qeo.20.1362614866490; Wed, 06 Mar 2013 16:07:46 -0800 (PST) Received: by 10.229.144.209 with HTTP; Wed, 6 Mar 2013 16:07:46 -0800 (PST) X-Originating-IP: [82.8.60.113] Received: by 10.229.144.209 with HTTP; Wed, 6 Mar 2013 16:07:46 -0800 (PST) In-Reply-To: References: Date: Thu, 7 Mar 2013 00:07:46 +0000 Message-ID: Subject: mkdir & open not setting sticky bit From: Nicholas Wilson To: "freebsd-hackers@freebsd.org" X-Gm-Message-State: ALoCoQm/6oHHFdU8kLpcLsFJuG55tNTBNrZYbdvwIueTSZGep4pLYTDMYbtXruWAh+U9G0uOUI1i Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 00:07:49 -0000 Hello, I'm guessing it's well known that BUGS in sticky(8) explains that BSD mkdir doesn't set the sticky bit. I'd like to fix that, but the fact that it's still there from historical BSD code suggests it's going to break something to change it (or surely it would have been done already!). I can't find a PR or any discussion over it though. It would be good to fix it, because of all the seven unixes our company supports, BSD's the only one with this quirk. Imagine if someone shipped a product with a security bug because directories weren't being created with the correct permissions? I might easily have done. If I wrote a patch for it, would I be wasting my time? The only difficulty I can imagine is changing semantics for existing applications, which seems extremely unlikely to cause breakage for this call. umask is ignored so doesn't complicate things; and POSIX & XSI explicitly allow both the current behaviour, and that implemented on every other Unix. Best, Nicholas From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 7 03:25:02 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 608CF763 for ; Thu, 7 Mar 2013 03:25:02 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-qe0-f49.google.com (mail-qe0-f49.google.com [209.85.128.49]) by mx1.freebsd.org (Postfix) with ESMTP id DD63CB6C for ; Thu, 7 Mar 2013 03:25:01 +0000 (UTC) Received: by mail-qe0-f49.google.com with SMTP id 1so7132qec.36 for ; Wed, 06 Mar 2013 19:25:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:subject:references:from:content-type:x-mailer :in-reply-to:message-id:date:to:content-transfer-encoding :mime-version:x-gm-message-state; bh=6p30LIlL+zYj6M2F5ma9yC9vVoTZK3NIGS1hfIvpTq8=; b=iyrI/cmHrkqONQAtYY5SRqKXR1kxCdSM6+IKH1wBjApv8rW35xecfyUONeuoGec4SL /ufBC8TKfYmZykXuKKaOHrWPqNiZ1cTxCNOMyUUhdmUsXZKWkqkJqthRN2bIaS/WXeDk D86NMtwf/xX3E20VEllbNh3B2e4Oz8DYI/OoNbvsNT7HUXZMX20t7pUNCInKulxF3IwO rwqij8g8C+iFLI7OQPNLf7UhAhijgA/rCHEF+S0eMCZWD60iro9ThTSbJgrYbqVOFAh4 /Z9rTRvafm8kXoDjM30zuq/G3xE4AanSI3bNIb6crouA9kyY5kVBBALqpvG5ElC4rRa1 DAvw== X-Received: by 10.224.108.4 with SMTP id d4mr27624501qap.83.1362626700810; Wed, 06 Mar 2013 19:25:00 -0800 (PST) Received: from [97.249.201.102] (102.sub-97-249-201.myvzw.com. [97.249.201.102]) by mx.google.com with ESMTPS id gw9sm681030qab.10.2013.03.06.19.24.55 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 06 Mar 2013 19:24:59 -0800 (PST) Subject: Re: gjournal +UFS - anyone actually use it? References: <15BA8CB5-A10A-4FA9-AC5F-1BA2B7720E5E@longcount.org> From: Mark Saad X-Mailer: iPhone Mail (10B146) In-Reply-To: Message-Id: <44621188-C3CF-44EC-BC94-CF85530C9423@longcount.org> Date: Wed, 6 Mar 2013 22:24:48 -0500 To: "freebsd-hackers@freebsd.org" Mime-Version: 1.0 (1.0) X-Gm-Message-State: ALoCoQmP0l3ax3sH8cYu08ADnq8KNYH2+LQyVz3+Dz7VYnXp4D/V1x0/JbVlx2/NYbSA5El07De0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 03:25:02 -0000 On Mar 6, 2013, at 5:55 AM, Wojciech Puchar = wrote: >> I was using it to store large MySQL myisam tables , speed was acceptable a= t the time . I never had any fs corruption and it worked as expected . >>=20 >> At the time I set it up I remember there was some chatter about how slow g= journal was compared to ufs with softupdate . >=20 > did some tests yesterday on 25GB partition. >=20 > i simulated journal on SSD using 2GB malloc backed ramdisk (md0). UFS part= ition mounted async as gjournal recommends. >=20 > test: unpacking kernel sources to multiple directories until disk gets ful= l >=20 > simulates write heavy I/O. >=20 > gjournal+SATA drive (seagate constellation ES 500GB) with write cache disa= bled - 14 minutes 20 seconds. >=20 > write cache enabled - 14 minutes 5 seconds (nearly no difference). >=20 > UFS+journalled softupdates, no gjournal, disk write cache on - 26 minutes 4= 4 seconds. disk write cache off - was too lazy to wait after an hour. >=20 > With gjournal it is not only faster, but it doesn't make other I/O activit= y crawling. >=20 Interesting I will have to try this; can you post the exact test steps . Als= o what type of controller were you using and what kernel / version . >=20 >>=20 >> Fast forward to today I almost always use ufs with softupdate journal , n= ew in FreeBSD 9.0 and available as a patch to 6.x, 7.x , and 8.x >=20 > The problem is as follows: >=20 > SU+J makes sure that metadata will get consistent. NOT DATA. And this is q= uite a mess if you get UPS failure under high load. >=20 > gjournal does journal everything. >=20 Not exactly, ufs mounted with default options insures data is written sync a= nd metadata asynchronous . Standard Softupdate (no journal) improves upon th= is by limiting what ops need to write to the disk. It had some short falls f= or edge case operations; which softupdate journal resolved by journaling the= metadata ops that were not protected / covered by standard softupdate . =20= See=20 http://jeffr-tech.livejournal.com/24357.html >> This is better supported now , as more people use it in new 9.x builds .= >>=20 >>> i think about journal on SSD. >>=20 >> I believe this is only and option in geom journal , >=20 > SSDs are not expensive today. i can get 128GB SSD and create 20GB journal j= ust to limit wear. and possibly use the rest of SSD to store read-intensive d= ata. >=20 I wonder if how trim / no trim effects the journal wear .=20 > the way gjournal writes to journal device (sequential) is very friendly fo= r SSD too. >=20 > Small SLC-based SSD would be best but i don't see anything on the market w= ith acceptable price for now. >=20 >> I am not sure if you can relocate a suj journal to an alternate disk. > no you can't. But still you will end with consistent metadata but not cons= istent data. I recovered it but still it took a time and lots of checking. >=20 >=20 >=20 >=20 > gjournal doesn't seem to be elegant in case of journal failure (i simulate= d it with forced removal of ramdisk with mdconfig). >=20 > TONS of messages in logs, but still - no data loss, just you have to shutd= own system, boot from pendrive, remove journal, fsck (just for sure), and th= en add journal again I would be careful of using the md for the journal . Something makes me thin= k it will play nicer when you remove that then a real failure . Try a USB s= tick for the journal; and pull it out while running your test case. That to= me seams evil enough to break this setup .=20 Let me know what happens . Also when testing su+j I ran the following test case . Extract ports via por= tsnap extract , build world with -j4 . Let the box warm up the yank the powe= r and then boot the box back up and see what happens .=20 -- Mark saad | mark.saad@longcount.org =20= From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 7 09:08:30 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 5441B89B for ; Thu, 7 Mar 2013 09:08:30 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id C578DCB4 for ; Thu, 7 Mar 2013 09:08:29 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r2798IYk009825; Thu, 7 Mar 2013 10:08:18 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r2798H4l009821; Thu, 7 Mar 2013 10:08:18 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Thu, 7 Mar 2013 10:08:17 +0100 (CET) From: Wojciech Puchar To: Mark Saad Subject: Re: gjournal +UFS - anyone actually use it? In-Reply-To: <44621188-C3CF-44EC-BC94-CF85530C9423@longcount.org> Message-ID: References: <15BA8CB5-A10A-4FA9-AC5F-1BA2B7720E5E@longcount.org> <44621188-C3CF-44EC-BC94-CF85530C9423@longcount.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Thu, 07 Mar 2013 10:08:18 +0100 (CET) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 09:08:30 -0000 > > Interesting I will have to try this; can you post the exact test steps . Also what type of controller were you using and what kernel / version . intel atom D525 builtin ahci0: port 0x20b8-0x20bf,0x20cc-0x20cf,0x20b0-0x20b7,0x20c8-0x20cb,0x20a0-0x20af mem 0xf0284000-0xf02843ff irq 18 at device 31.2 on pci0 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ATA-8 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 (BTW - no idea why it previously was known as ad4 as i've never ever run this machine without AHCI driver ;) create partition. copy kernel sources tarball here. now on that partition rm -rf * then write a script test.sh #!/usr/local/bin/bash a=0 while [ $a -lt 12 ];do mkdir $a cd $a a=$[a+1] tar xpf ../yourtarball.tar.gz cd .. done run it with (after chmod 700) time ./test.sh rerun a test on different conditions. >> >> SU+J makes sure that metadata will get consistent. NOT DATA. And this is quite a mess if you get UPS failure under high load. >> >> gjournal does journal everything. >> > > Not exactly, ufs mounted with default options insures data is written sync and metadata asynchronous . Standard Softupdate (no journal) improves upon this by limiting what ops need to write to the disk. It had some short falls for edge case operations; which softupdate journal resolved by journaling the metadata ops that were not protected / covered by standard softupdate . > > See > http://jeffr-tech.livejournal.com/24357.html gjournal recommends using async, because it takes care of the rest. my tests using pendrive as log device (slow but OK) confirmed my opinion, at least roughly. >> SSDs are not expensive today. i can get 128GB SSD and create 20GB journal just to limit wear. and possibly use the rest of SSD to store read-intensive data. >> > > I wonder if how trim / no trim effects the journal wear . not at all. gjournal writes sequentially. best case for SSD - just write subsequent flash blocks and older gets freed completely and erase it. >> gjournal doesn't seem to be elegant in case of journal failure (i simulated it with forced removal of ramdisk with mdconfig). >> >> TONS of messages in logs, but still - no data loss, just you have to shutdown system, boot from pendrive, remove journal, fsck (just for sure), and then add journal again > > > I would be careful of using the md for the journal . > >Something makes me think it will play nicer when you remove that then a real failure . Try a USB stick for the journal; tried. same results. md was only for testing, doesn't make any sense in production setup as it's volatile. > Also when testing su+j I ran the following test case . Extract ports via portsnap extract , build world with -j4 . Let the box warm up the yank the power and then boot the box back up and see what happens . and? didn't you end with empty - but existing - object files that make things it is properly compiled programs? From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 7 11:24:42 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id EF2FB94E for ; Thu, 7 Mar 2013 11:24:42 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 533ED26B for ; Thu, 7 Mar 2013 11:24:42 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r27BOW7L010519 for ; Thu, 7 Mar 2013 12:24:32 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r27BOWhP010516 for ; Thu, 7 Mar 2013 12:24:32 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Thu, 7 Mar 2013 12:24:32 +0100 (CET) From: Wojciech Puchar To: freebsd-hackers@freebsd.org Subject: lots of network interfaces Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Thu, 07 Mar 2013 12:24:32 +0100 (CET) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 11:24:43 -0000 are there any scalability limits in case of lots of network interfaces. i am asking for ca 800 tun(4) interfaces active but no more than 250Mbit/s over them total. From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 7 11:59:46 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9236C202; Thu, 7 Mar 2013 11:59:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A230D3E0; Thu, 7 Mar 2013 11:59:45 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA17964; Thu, 07 Mar 2013 13:59:43 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UDZU7-0009rf-09; Thu, 07 Mar 2013 13:59:43 +0200 Message-ID: <5138812D.5060304@FreeBSD.org> Date: Thu, 07 Mar 2013 13:59:41 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org Subject: dtrace -c patch X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 11:59:46 -0000 $ dtrace -n 'blah blah' -c 'touch /' dtrace: failed to control pid 16473: process exited with status 0 I think that we need the following change: --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c @@ -1115,7 +1115,7 @@ alloc: #if defined(sun) dtp->dt_prcmode = DT_PROC_STOP_PREINIT; #else - dtp->dt_prcmode = DT_PROC_STOP_MAIN; + dtp->dt_prcmode = DT_PROC_STOP_POSTINIT; #endif dtp->dt_linkmode = DT_LINK_KERNEL; dtp->dt_linktype = DT_LTYP_ELF; Rationale: - we don't use DT_PROC_STOP_PREINIT like solaris does, because for some reason that I haven't investigated we the following line ifdef-ed out in dt_proc_attach: dt_proc_rdwatch(dpr, RD_PREINIT, "RD_PREINIT"); - 'main' symbol can't always be reliably resolved: $ readelf -a -W /usr/bin/touch | fgrep main $ So, it seems that "postinit" is where we can reliable catch a process. What do you think? -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 7 18:17:06 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id F3037532 for ; Thu, 7 Mar 2013 18:17:06 +0000 (UTC) (envelope-from erdgeist@erdgeist.org) Received: from elektropost.org (elektropost.org [217.13.206.130]) by mx1.freebsd.org (Postfix) with ESMTP id 50C95ABC for ; Thu, 7 Mar 2013 18:17:05 +0000 (UTC) Received: (qmail 86772 invoked from network); 7 Mar 2013 18:16:59 -0000 Received: from elektropost.org (HELO elektropost.org) (erdgeist@erdgeist.org) by elektropost.org with CAMELLIA256-SHA encrypted SMTP; 7 Mar 2013 18:16:59 -0000 Message-ID: <5138D99E.8010700@erdgeist.org> Date: Thu, 07 Mar 2013 19:17:02 +0100 From: Dirk Engling User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130216 Thunderbird/17.0.3 MIME-Version: 1.0 CC: freebsd-hackers@freebsd.org Subject: Re: pop filters from kqueue References: <513622F2.20902@mu.org> In-Reply-To: <513622F2.20902@mu.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 18:17:07 -0000 On 05.03.13 17:53, Alfred Perlstein wrote: Alfred, > I'm not sure if kqueue support this, however adding such a facility > might be OK. thanks for your reply. What would be the correct place to suggest such a change? > The only pain here is that it requires managing a doubly linked list and > additional pointer dereferences. Which seems extra absurd in the light of a performant implementation already handling all this - within the kqueue. And it still does not solve my problem that for every operation on my allocation I need to update pointers both in the kqueue and in my linked list. Regards, erdgeist From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 7 23:41:19 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BFFEA4CF for ; Thu, 7 Mar 2013 23:41:19 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) by mx1.freebsd.org (Postfix) with ESMTP id 511C9AE7 for ; Thu, 7 Mar 2013 23:41:19 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id d46so331499wer.17 for ; Thu, 07 Mar 2013 15:41:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=fBJ9sGNMugVCcfILkf4XlJHKMZXoaC8b9PWIV8qQEFs=; b=n47fpCwXC8QwSnssYDw2D8gHkFhsYlbaA69dOTtHoIVi+aOewdU/2bA51COxHTmGEv HSBF78OjPhtxNfgx53MDmAqhQIfvVYgvJiG9/e9AW20b9pIWoD6ZqEPAkNQ8axGVFY0P EVOO/tCoP8hayUHILNV1SnZQDiYVRzH56FjM6Pwy4yni9ZmFle85aLpQNJZEpSG8fGK4 Mn2+OFH8f6VBKrF9fUV1JJ10sgFlUcES3VCU2eLFLdMGKzm/MTwCCa+9mA0LQzgRZGPd TCziRzl9XuCtGszvNErhFDJYPR/hPuybfim/ra1fhN1mwmJBlVxatdSbN8Fp2Zrp6uMh fWKg== MIME-Version: 1.0 X-Received: by 10.181.12.5 with SMTP id em5mr208634wid.24.1362699677076; Thu, 07 Mar 2013 15:41:17 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.217.51.2 with HTTP; Thu, 7 Mar 2013 15:41:16 -0800 (PST) In-Reply-To: References: Date: Thu, 7 Mar 2013 15:41:16 -0800 X-Google-Sender-Auth: 79AixClTsOy-wnIj57R4zttfw3o Message-ID: Subject: Re: lots of network interfaces From: Adrian Chadd To: Wojciech Puchar Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 23:41:19 -0000 On 7 March 2013 03:24, Wojciech Puchar wrote: > are there any scalability limits in case of lots of network interfaces. > i am asking for ca 800 tun(4) interfaces active but no more than 250Mbit/s > over them total. Do you hit CPU limits at that point? Adrian From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 8 00:12:10 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D9DAA2D4 for ; Fri, 8 Mar 2013 00:12:10 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) by mx1.freebsd.org (Postfix) with ESMTP id AC2AFCBB for ; Fri, 8 Mar 2013 00:12:10 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.16]) by ltcfislmsgpa06.fnfis.com (8.14.5/8.14.5) with ESMTP id r280C9UJ002473 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Thu, 7 Mar 2013 18:12:09 -0600 Received: from dtwin (10.242.182.25) by smtp.fisglobal.com (10.132.206.16) with Microsoft SMTP Server (TLS) id 14.2.309.2; Thu, 7 Mar 2013 18:12:08 -0600 From: Sender: Devin Teske To: "'Wojciech Puchar'" , References: In-Reply-To: Subject: RE: lots of network interfaces Date: Thu, 7 Mar 2013 16:11:37 -0800 Message-ID: <031201ce1b91$841b90d0$8c52b270$@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQKpwYVpQ9wq31fj1Xr8NqKcq8vI85bjnyWg Content-Language: en-us X-Originating-IP: [10.242.182.25] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2013-03-07_09:2013-03-07,2013-03-07,1970-01-01 signatures=0 Cc: devin.teske@fisglobal.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 00:12:10 -0000 I decided to stress-test the netgraph(4) subsystem one day and was able to create 65530 interfaces before it produced an error, refusing to create another. At that point, the system was still usable, but... It took over an hour for ifconfig to list all the interfaces. Simply typing "ifconfig" with no arguments and pressing ENTER would start spewing information on-screen for over an hour before it finished. (so I'd say that there could be some optimizations made; but nonetheless impressive that the system was still very usable at that point, ifconfig aside). -- Devin > -----Original Message----- > From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd- > hackers@freebsd.org] On Behalf Of Wojciech Puchar > Sent: Thursday, March 07, 2013 3:25 AM > To: freebsd-hackers@freebsd.org > Subject: lots of network interfaces > > are there any scalability limits in case of lots of network interfaces. > i am asking for ca 800 tun(4) interfaces active but no more than 250Mbit/s > over them total. > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 8 16:15:11 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 123EED67; Fri, 8 Mar 2013 16:15:11 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 83F97F3B; Fri, 8 Mar 2013 16:15:10 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r28GF0lb049963; Fri, 8 Mar 2013 17:15:00 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r28GExOE049960; Fri, 8 Mar 2013 17:15:00 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Fri, 8 Mar 2013 17:14:59 +0100 (CET) From: Wojciech Puchar To: Adrian Chadd Subject: Re: lots of network interfaces In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Fri, 08 Mar 2013 17:15:00 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 16:15:11 -0000 >> i am asking for ca 800 tun(4) interfaces active but no more than 250Mbit/s >> over them total. > > Do you hit CPU limits at that point? i am not because i don't do this now, and ask BEFORE planning. > > > > Adrian > > From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 8 16:51:04 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2F37A756; Fri, 8 Mar 2013 16:51:04 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id CEA5A273; Fri, 8 Mar 2013 16:51:03 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:8571:2d32:217f:d124]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 7713E4AC57; Fri, 8 Mar 2013 20:51:02 +0400 (MSK) Date: Fri, 8 Mar 2013 20:50:57 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <891734798.20130308205057@serebryakov.spb.ru> To: Wojciech Puchar Subject: Re: lots of network interfaces In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Adrian Chadd X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 16:51:04 -0000 Hello, Wojciech. You wrote 8 =D0=BC=D0=B0=D1=80=D1=82=D0=B0 2013 =D0=B3., 20:14:59: WP> i am not because i don't do this now, and ask BEFORE planning. I know ISP, who uses FreeBSD with mpd5 as BRAS for PPPoE. They uses ngX, not tunX, but as far as I understand, their systems has thousands such interfaces (~2500 per machine, like 4xCore Supermicro platform, nothing like Really Big Iron). --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 8 16:57:06 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A96EF8E7; Fri, 8 Mar 2013 16:57:06 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 307AE2D9; Fri, 8 Mar 2013 16:57:05 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r28GuuVo050149; Fri, 8 Mar 2013 17:56:56 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r28Guuk4050146; Fri, 8 Mar 2013 17:56:56 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Fri, 8 Mar 2013 17:56:56 +0100 (CET) From: Wojciech Puchar To: Lev Serebryakov Subject: Re: lots of network interfaces In-Reply-To: <891734798.20130308205057@serebryakov.spb.ru> Message-ID: References: <891734798.20130308205057@serebryakov.spb.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Fri, 08 Mar 2013 17:56:56 +0100 (CET) Cc: freebsd-hackers@freebsd.org, Adrian Chadd X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 16:57:06 -0000 > You wrote 8 ????? 2013 ?., 20:14:59: > > WP> i am not because i don't do this now, and ask BEFORE planning. > I know ISP, who uses FreeBSD with mpd5 as BRAS for PPPoE. They uses exactly what i want, just with vtun. So it will go fine. thank you. From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 8 18:19:39 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BB768F98; Fri, 8 Mar 2013 18:19:39 +0000 (UTC) (envelope-from freebsd-lists@be-well.ilk.org) Received: from be-well.ilk.org (be-well.ilk.org [23.30.133.173]) by mx1.freebsd.org (Postfix) with ESMTP id 93BEBCC9; Fri, 8 Mar 2013 18:19:39 +0000 (UTC) Received: from lowell-desk.lan (lowell-desk.lan [172.30.250.41]) by be-well.ilk.org (Postfix) with ESMTP id 91A1E33C1D; Fri, 8 Mar 2013 13:06:18 -0500 (EST) Received: by lowell-desk.lan (Postfix, from userid 1147) id E0B0439841; Fri, 8 Mar 2013 13:06:16 -0500 (EST) From: Lowell Gilbert To: Subject: Re: lots of network interfaces References: <031201ce1b91$841b90d0$8c52b270$@freebsd.org> Date: Fri, 08 Mar 2013 13:06:16 -0500 In-Reply-To: <031201ce1b91$841b90d0$8c52b270$@freebsd.org> (dteske@freebsd.org's message of "Thu, 7 Mar 2013 16:11:37 -0800") Message-ID: <44y5dx3gx3.fsf@lowell-desk.lan> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain Cc: 'Wojciech Puchar' , devin.teske@fisglobal.com, freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: freebsd-hackers@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 18:19:39 -0000 writes: > I decided to stress-test the netgraph(4) subsystem one day and was able to > create 65530 interfaces before it produced an error, refusing to create another. > > At that point, the system was still usable, but... > > It took over an hour for ifconfig to list all the interfaces. Simply typing > "ifconfig" with no arguments and pressing ENTER would start spewing information > on-screen for over an hour before it finished. (so I'd say that there could be > some optimizations made; but nonetheless impressive that the system was still > very usable at that point, ifconfig aside). The interfaces are a linked list, plus there's a separate kernel dive for each interface. The list (as opposed to individual interfaces) is really only accessed from userland, so optimizing these operations would need a really good use case to be worthwhile. The current implementation is not just very well known, it's good for hundreds to thousands of interfaces..For the output of ifconfig alone, it might be possible to grab more information in the original getifaddrs() (which makes a copy of the interface list), but that would make other operations with that function more expensive. From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 9 07:16:36 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B150971B; Sat, 9 Mar 2013 07:16:36 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 27FDEE55; Sat, 9 Mar 2013 07:16:35 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r297GL9p072005; Sat, 9 Mar 2013 08:16:21 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r297GKKB072002; Sat, 9 Mar 2013 08:16:20 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Sat, 9 Mar 2013 08:16:20 +0100 (CET) From: Wojciech Puchar To: freebsd-hackers@freebsd.org Subject: Re: lots of network interfaces In-Reply-To: <44y5dx3gx3.fsf@lowell-desk.lan> Message-ID: References: <031201ce1b91$841b90d0$8c52b270$@freebsd.org> <44y5dx3gx3.fsf@lowell-desk.lan> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Sat, 09 Mar 2013 08:16:21 +0100 (CET) Cc: dteske@freebsd.org, devin.teske@fisglobal.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 07:16:36 -0000 >> very usable at that point, ifconfig aside). > > The interfaces are a linked list, plus there's a separate kernel dive > for each interface. The list (as opposed to individual interfaces) is i don't care how fast ifconfig displays them, just for how fast things like routing or ipfw rule like "via tun196" would work. if it is hashed somehow - fine. if not - very bad. From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 9 14:23:20 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 35DDE6C4; Sat, 9 Mar 2013 14:23:20 +0000 (UTC) (envelope-from freebsd-lists@be-well.ilk.org) Received: from be-well.ilk.org (be-well.ilk.org [23.30.133.173]) by mx1.freebsd.org (Postfix) with ESMTP id E3DD639A; Sat, 9 Mar 2013 14:23:19 +0000 (UTC) Received: from lowell-desk.lan (lowell-desk.lan [172.30.250.41]) by be-well.ilk.org (Postfix) with ESMTP id A001133C1D; Sat, 9 Mar 2013 09:23:07 -0500 (EST) Received: by lowell-desk.lan (Postfix, from userid 1147) id BB9483983C; Sat, 9 Mar 2013 09:23:06 -0500 (EST) From: Lowell Gilbert To: Wojciech Puchar Subject: Re: lots of network interfaces References: <031201ce1b91$841b90d0$8c52b270$@freebsd.org> <44y5dx3gx3.fsf@lowell-desk.lan> Date: Sat, 09 Mar 2013 09:23:06 -0500 In-Reply-To: (Wojciech Puchar's message of "Sat, 9 Mar 2013 08:16:20 +0100 (CET)") Message-ID: <44boasodo5.fsf@lowell-desk.lan> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain Cc: freebsd-hackers@freebsd.org, dteske@freebsd.org, devin.teske@fisglobal.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: freebsd-hackers@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 14:23:20 -0000 Wojciech Puchar writes: >>> very usable at that point, ifconfig aside). >> >> The interfaces are a linked list, plus there's a separate kernel dive >> for each interface. The list (as opposed to individual interfaces) is > > i don't care how fast ifconfig displays them, just for how fast things > like routing or ipfw rule like "via tun196" would work. > > if it is hashed somehow - fine. > if not - very bad. Not an issue, as I explained in a part of the message that you clipped out. From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 9 22:41:11 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E2281225 for ; Sat, 9 Mar 2013 22:41:11 +0000 (UTC) (envelope-from ali_mousa_zy@yahoo.com) Received: from nm28-vm0.bullet.mail.bf1.yahoo.com (nm28-vm0.bullet.mail.bf1.yahoo.com [98.139.213.149]) by mx1.freebsd.org (Postfix) with ESMTP id 8B63FDC2 for ; Sat, 9 Mar 2013 22:41:11 +0000 (UTC) Received: from [98.139.215.140] by nm28.bullet.mail.bf1.yahoo.com with NNFMP; 09 Mar 2013 22:41:05 -0000 Received: from [98.139.212.219] by tm11.bullet.mail.bf1.yahoo.com with NNFMP; 09 Mar 2013 22:41:05 -0000 Received: from [127.0.0.1] by omp1028.mail.bf1.yahoo.com with NNFMP; 09 Mar 2013 22:41:05 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 697732.62354.bm@omp1028.mail.bf1.yahoo.com Received: (qmail 14970 invoked by uid 60001); 9 Mar 2013 22:41:05 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1362868865; bh=zWFN6xjN38J5mzFgeyBU7jZgmpWmxwxu+B+zk6vqrV4=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=xePv/OvBfVEarUb3+1MJYD7BB4eiG6olNkyPneC65aObuGmDDWS0wvbTSH08U6DYlKkV8+lMrz91hyRSWJVW819OG6s3xO4paXiHbDzzYKxGUoJ38s1TwdJuBXXcK8YayN33J5xTZbZPi4a3TsM8I6S+m+TpYnjvWyj7ZvqZ45o= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=i+ngUjcYoLkPf7g7NpBKkb9qml8ttaAh2pxtmuBo4JCdwxOWxsQ6XuIS/8/QtpA6z4IRMtF7YW1SZTp27TUqtxWxFblrP/KqRU957mRsCXIOWtsqB/rI3hnYkVgFrgOdXHAiCIkm1daiCWKUyNggFwdSM9ue8eWAmtGLSQ/Cmvs=; X-YMail-OSG: 0ORf30wVM1kESh52J_D4DWpRI8mfIMZqddUpa93Y7hB8EV5 WNnRNSyLoRUojy_bB.elTTfqll0l9b1Cu4sGsxOw8kbH0xJnpDLPlk4AbU1u 6VtnyjyY5_yr5ng4NRMYqoQwfbGeyQcu2fFdR55FiYwz5gAoW7e3Y5M5sqM2 Anz_KMZKphDe9FrqQ2OAs_57K62KWPvCBziLK0YzClk8yAv.VxliLyWe2RfV LPaIn.IrAK7trFCB86xAMBmCZgS9AifkpbL7UAakwMASMOoHMB0FGH2s8ZtB nDf0SSsRjDb0qG0k7CPmO.8ZcPkmgztvxav9IEBDBi81YApFxPNPM1jEIcit g_j2ytXZxAJfzSHl85nUpqnVhcaa72H.Alsx9d9XvF6z8fy2gYo43yoGjdZt PsBYM9AL7vb5RPhQh5KGO1oX9yG8Wp3tofle2sdCrd9un6z9K1EP7emwZNPE VpNna7G8HVkpk9fJtqR1p.kdqauW4LOCelKeG_seYIXKJS8jaVn10BvcjuiP cwwweg3M2FSprYS.DptpNGtdPLs3sG6wOVYzLb6br0ghnT5BOJ5ULFE5eP2R z6fNtWDrWknOosmKIG0iNZ2KjSUk- Received: from [46.185.160.199] by web140001.mail.bf1.yahoo.com via HTTP; Sat, 09 Mar 2013 14:41:05 PST X-Rocket-MIMEInfo: 001.001, RG9lcyBhbnkgYm9keSBhZ3JlZSA_IGh0dHA6Ly9mb3J1bXMuZnJlZWJzZC5vcmcvc2hvd3RocmVhZC5waHA_cD0yMTI0MjMjcG9zdDIxMjQyMyA_IAoKClJlZ2FyZHMKCsKgCkFsaQEwAQEBAQ-- X-Mailer: YahooMailWebService/0.8.135.514 Message-ID: <1362868865.9698.YahooMailNeo@web140001.mail.bf1.yahoo.com> Date: Sat, 9 Mar 2013 14:41:05 -0800 (PST) From: ali mousa Subject: Does any body agree ? To: "freebsd-hackers@freebsd.org" MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: ali mousa List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 22:41:11 -0000 Does any body agree ? http://forums.freebsd.org/showthread.php?p=3D212423#p= ost212423 ? =0A=0A=0ARegards=0A=0A=A0=0AAli From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 9 23:03:14 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 398ADB41 for ; Sat, 9 Mar 2013 23:03:14 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) by mx1.freebsd.org (Postfix) with ESMTP id A489EE8C for ; Sat, 9 Mar 2013 23:03:13 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id d46so2380346wer.3 for ; Sat, 09 Mar 2013 15:03:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=07uIFb14sSYqdKT51YLvYNEtRF9wzCWoY0B8uyylpGM=; b=J4be4ioAJFb4+3CXT72xs60fKtSNCaWeJ3FPZNn7nB/t75hZvoBpxmz3vlLSUo+GMc F4ja/VMBKJuhx6JE/bW7r/TpwQDTtH7d891A2byFTDZ3tfdu6Z3vchRqhWHMtB3lEkRR kWgqUFzWp0Sx2FrKFJUKE6ensLHhyWVHtDKOZicMOJVjlu3SUHC9dkZ9LhuGkNskHfKt 81LCy92xsOPUMsUs+6E98/EL80HmF/HMIQzAA66xRJZU7PE0JzH4YfAe7XeOtOE7BkPI +55qxTMlq9T9cVbShUtcFPPM6ipNtF8lVN9ya3wdpACAqU0ICubfeftxXM9oOzSawrFK xr7Q== MIME-Version: 1.0 X-Received: by 10.194.87.229 with SMTP id bb5mr11564344wjb.32.1362870192728; Sat, 09 Mar 2013 15:03:12 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.111.201 with HTTP; Sat, 9 Mar 2013 15:03:12 -0800 (PST) In-Reply-To: <1362868865.9698.YahooMailNeo@web140001.mail.bf1.yahoo.com> References: <1362868865.9698.YahooMailNeo@web140001.mail.bf1.yahoo.com> Date: Sat, 9 Mar 2013 15:03:12 -0800 X-Google-Sender-Auth: zPjp49yKzxGWXUGVoaKtlSSXRu0 Message-ID: Subject: Re: Does any body agree ? From: Adrian Chadd To: ali mousa Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 23:03:14 -0000 ... you need to post a more useful/descrptive title and/or body in your request. Most people won't look at the post. :-) Adrian On 9 March 2013 14:41, ali mousa wrote: > Does any body agree ? http://forums.freebsd.org/showthread.php?p=212423#post212423 ? > > > Regards > > > Ali > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 9 23:06:51 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D4350F90; Sat, 9 Mar 2013 23:06:51 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-ve0-f169.google.com (mail-ve0-f169.google.com [209.85.128.169]) by mx1.freebsd.org (Postfix) with ESMTP id 726C1EB6; Sat, 9 Mar 2013 23:06:51 +0000 (UTC) Received: by mail-ve0-f169.google.com with SMTP id 15so2018111vea.28 for ; Sat, 09 Mar 2013 15:06:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=5GmJl6WovOe1HASuCk9ypqwIyNYoa65STEYFEVTyZTs=; b=Mi3rE2aoHUjmgyEe+6zT49jrEct+Vu4friVt5BXGtBSKTNxjGJ3/DhmVqPRS5VHUyQ oCB2lMHIxb/7rWynTx+5sYTq7uykFrS9rp9HV7yd5/QZlA6DK2iCNIISPpkigcAEHg7l OteMpADYNZ+02ZpVyw0z21H4uU4q4f4/icBwRuAkqkMx+EkA0BYrCFnfK1bec+ldz9Up t+N1e06aKiMU/L6Okq3JMOkSW7JNyoXkBfhSke7wC65gua+sHP4ARpk8kKmu7gj6D5u+ eAzj+YeLruDd4eZZn/RRykGDsYziqSRvnXckdAEIGgLqe2G288UWuANriTorXvTMD3IV rS6w== MIME-Version: 1.0 X-Received: by 10.220.242.73 with SMTP id lh9mr2782625vcb.49.1362870405752; Sat, 09 Mar 2013 15:06:45 -0800 (PST) Received: by 10.58.220.33 with HTTP; Sat, 9 Mar 2013 15:06:45 -0800 (PST) In-Reply-To: References: <1362868865.9698.YahooMailNeo@web140001.mail.bf1.yahoo.com> Date: Sat, 9 Mar 2013 15:06:45 -0800 Message-ID: Subject: Re: Does any body agree ? From: Mehmet Erol Sanliturk To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" , ali mousa X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 23:06:51 -0000 On Sat, Mar 9, 2013 at 3:03 PM, Adrian Chadd wrote: > ... you need to post a more useful/descrptive title and/or body in > your request. Most people won't look at the post. :-) > > > > Adrian > > > On 9 March 2013 14:41, ali mousa wrote: > > Does any body agree ? > http://forums.freebsd.org/showthread.php?p=212423#post212423 ? > > > > > > Regards > > > > > > Ali > > _ > Dear Adrian , You are really right , and correct . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 9 23:12:09 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id EEDC319A; Sat, 9 Mar 2013 23:12:09 +0000 (UTC) (envelope-from prvs=1780d9d615=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 6D80BEE4; Sat, 9 Mar 2013 23:12:08 +0000 (UTC) Received: from r2d2 ([46.65.172.4]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50002641790.msg; Sat, 09 Mar 2013 23:11:07 +0000 X-Spam-Processed: mail1.multiplay.co.uk, Sat, 09 Mar 2013 23:11:07 +0000 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 46.65.172.4 X-Return-Path: prvs=1780d9d615=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Mehmet Erol Sanliturk" , "Adrian Chadd" References: <1362868865.9698.YahooMailNeo@web140001.mail.bf1.yahoo.com> Subject: Re: Does any body agree ? Date: Sat, 9 Mar 2013 23:11:06 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-hackers@freebsd.org, ali mousa X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 23:12:10 -0000 ----- Original Message ----- From: "Mehmet Erol Sanliturk" To: "Adrian Chadd" Cc: ; "ali mousa" Sent: Saturday, March 09, 2013 11:06 PM Subject: Re: Does any body agree ? > On Sat, Mar 9, 2013 at 3:03 PM, Adrian Chadd wrote: > >> ... you need to post a more useful/descrptive title and/or body in >> your request. Most people won't look at the post. :-) >> >> >> >> Adrian >> >> >> On 9 March 2013 14:41, ali mousa wrote: >> > Does any body agree ? >> http://forums.freebsd.org/showthread.php?p=212423#post212423 ? >> > >> > >> > Regards >> > >> > >> > Ali >> > _ >> > > > Dear Adrian , > > You are really right , and correct . > > Thank you very much . You may wish to know I asked the ports security team if they where working on this and they are. I've not seen it hit svn yet so hopefully this will happen soon. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 9 23:16:39 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 072533A8 for ; Sat, 9 Mar 2013 23:16:39 +0000 (UTC) (envelope-from ali_mousa_zy@yahoo.com) Received: from nm18-vm0.bullet.mail.bf1.yahoo.com (nm18-vm0.bullet.mail.bf1.yahoo.com [98.139.213.138]) by mx1.freebsd.org (Postfix) with SMTP id 84DCEF0B for ; Sat, 9 Mar 2013 23:16:38 +0000 (UTC) Received: from [98.139.212.148] by nm18.bullet.mail.bf1.yahoo.com with NNFMP; 09 Mar 2013 23:16:32 -0000 Received: from [98.139.212.230] by tm5.bullet.mail.bf1.yahoo.com with NNFMP; 09 Mar 2013 23:16:32 -0000 Received: from [127.0.0.1] by omp1039.mail.bf1.yahoo.com with NNFMP; 09 Mar 2013 23:16:32 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 31560.53789.bm@omp1039.mail.bf1.yahoo.com Received: (qmail 50719 invoked by uid 60001); 9 Mar 2013 23:16:31 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1362870991; bh=F4e3sTkHWg77m8VISBWsqFWRkIGv7LIR8KUYMPJh82U=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=k2N1EbSVxtDJrm8ZE/ZhUuYzqSFOjhOdCSyQV7jVRwC2SI9sqwnDQxVq/Mg4BvTIOQq2LF7isEoEe/Z6E4s5VMluQHMtHe6zpmDS1QajyaZpD+/CQADLrbgFSXOEPhDU+HST0e+sGOiJkJwdTnvXUxXU6f1qofopDwsctAP2kBU= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=aMcHKv/G5lneTlO/Npx2qXTtICkB7wsMVJDLyh0pb5cC8fkF2A+/oxeivlJsKwFEOp9I8Zk2mz50e1DTMZiU9qPTLNCoMRH+wBp+lcwZ4j76vP7z+Y+eSn0F50nNd9RCIYI4YAbCEYMfTs7gPQcDEBZ/SMANv25UvY1lLABTnIs=; X-YMail-OSG: njSl7UMVM1laOoa.WnVfWzR0c9EDlYXZDIjpjUuzHsgFXhi hwA8cU6QvGOoLXNDHPNCK1PXAqw3EuQ2.HpFDCC_cPGlcfUVFHvlIaZykV77 sUNkHu3VA3_2zphoh1imVHkn9SOnxPXOefOSOzemu27A8W1ye7ZCuIZXzqgF Lt4nfZ6qQ2GTI3TtSTKtR_6rlQpYhwhFYLbkjHDj2NXScfezcNcHMeNPQNvY YbqQTtTh3jfVTJCCoEfEkCbyySV8ku5ngnTwzSHXM6XEAqceazHh9nZ3gM6q _Io_r9R534.j6zpiJKcYTh5peLkwpyGj10nnaIXHgg0Z4GRhcqxgFjmAaECF qgT8xWrZcOgMqutROhvPjOLLA90XzOKGABqgeuwZdsHw7yasxWfFgAM7xGHP pjlQn6ObOP8AQYKe1uAfFmuHqlbmqyb0lRz.hVgmUQE_zC2E1q458cofP66B Z1APms3S2B9VxY_.m0UpWXr1uakrgxtQhJ8_75LaVP4yOHBQaaFKOJw6vK.i JsTMbI.RBaXOp2G31kmzj0I5JuEumFkuZ399U5OB4o3BWS0NdGy1eL63ynVc rtV89YCK6iC3qsKr1F8MWJTbhlYR1dXLsUQGGmOnDMVKotWlc Received: from [46.185.160.199] by web140002.mail.bf1.yahoo.com via HTTP; Sat, 09 Mar 2013 15:16:31 PST X-Rocket-MIMEInfo: 001.001, SGVsbG8sIAoKSSBoYXZlIHJlYWQgdGhpcyBpbiBhIGJvb2sgY2FsbGVkIAoiCsKgwqDCoCBGcmVlQlNEIERldmljZSBEcml2ZXJzCgkqIEJ5OiBKb3NlcGggS29uZwoJKiBQdWJsaXNoZXI6IE5vIFN0YXJjaCBQcmVzcwoJKiBQdWIuIERhdGU6IE1heSAgNSwgMjAxMgoJKiBQcmludCBJU0JOLTEzOiA5NzgtMS01OTMyNy0yMDQtMwoJKiBQYWdlcyBpbiBQcmludCBFZGl0aW9uOiAzNTIKCSogIgrCoApBbGkKQWxyYWhhaGxlaCAKdGVjaCB5YWhvbywgc29mdHdhcmUgYXBwcyBkZXYgZW5nLAoKd29yayBlbWFpbCABMAEBAQE- X-Mailer: YahooMailWebService/0.8.135.514 References: <1362868865.9698.YahooMailNeo@web140001.mail.bf1.yahoo.com> Message-ID: <1362870991.46744.YahooMailNeo@web140002.mail.bf1.yahoo.com> Date: Sat, 9 Mar 2013 15:16:31 -0800 (PST) From: ali mousa Subject: Re: Does any body agree ? To: Steven Hartland , Mehmet Erol Sanliturk , Adrian Chadd In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: ali mousa List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 23:16:39 -0000 Hello, =0A=0AI have read this in a book called =0A"=0A=A0=A0=A0 FreeBSD Dev= ice Drivers=0A=09* By: Joseph Kong=0A=09* Publisher: No Starch Press=0A=09*= Pub. Date: May 5, 2012=0A=09* Print ISBN-13: 978-1-59327-204-3=0A=09* Pag= es in Print Edition: 352=0A=09* "=0A=A0=0AAli=0AAlrahahleh =0Atech yahoo, s= oftware apps dev eng,=0A=0Awork email :amousa@yahoo-inc.com=0Apersonal emai= l: ali_mousa_zy@yahoo.com=0Adirect +(962) 65506933=A0=A0=A0 mobile +(962) 7= 77465533=0A=0Aal husseini building 141 mecca street, amman, 1114, jo=0Aphon= e (408) 349 3300=A0=A0=A0 fax (408) 349 3301=0A=A0=A0=0A=0A________________= ________________=0A From: Steven Hartland =0ATo: M= ehmet Erol Sanliturk ; Adrian Chadd =0ACc: freebsd-hackers@freebsd.org; ali mousa =0ASent: Sunday, March 10, 2013 1:11 AM=0ASubject: Re: Does any body agr= ee ?=0A =0A=0A----- Original Message ----- =0AFrom: "Mehmet Erol Sanliturk"= =0ATo: "Adrian Chadd" =0ACc: = ; "ali mousa" =0ASent:= Saturday, March 09, 2013 11:06 PM=0ASubject: Re: Does any body agree ?=0A= =0A=0A> On Sat, Mar 9, 2013 at 3:03 PM, Adrian Chadd w= rote:=0A> =0A>> ... you need to post a more useful/descrptive title and/or = body in=0A>> your request. Most people won't look at the post. :-)=0A>>=0A>= >=0A>>=0A>> Adrian=0A>>=0A>>=0A>> On 9 March 2013 14:41, ali mousa wrote:=0A>> > Does any body agree ?=0A>> http://forums.fre= ebsd.org/showthread.php?p=3D212423#post212423 ?=0A>> >=0A>> >=0A>> > Regard= s=0A>> >=0A>> >=0A>> > Ali=0A>> > _=0A>>=0A> =0A> =0A> Dear Adrian ,=0A> = =0A> You are really right , and correct .=0A> =0A> Thank you very much .=0A= =0AYou may wish to know I asked the ports security team if they where=0Awor= king on this and they are. I've not seen it hit svn yet so hopefully=0Athis= will happen soon.=0A=0A=A0 =A0 Regards=0A=A0 =A0 Steve=0A=0A=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0AThis e.mail is = private and confidential between Multiplay (UK) Ltd. and the person or enti= ty to whom it is addressed. In the event of misdirection, the recipient is = prohibited from using, copying, printing or otherwise disseminating it or a= ny information contained in it. =0A=0AIn the event of misdirection, illegib= le or incomplete transmission please telephone +44 845 868 1337=0Aor return= the E.mail to postmaster@multiplay.co.uk.