From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 01:08:28 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1CC34C6E for ; Sun, 4 Nov 2012 01:08:28 +0000 (UTC) (envelope-from tzabal@it.teithe.gr) Received: from alpha.it.teithe.gr (alpha.it.teithe.gr [195.251.240.232]) by mx1.freebsd.org (Postfix) with ESMTP id 84C988FC12 for ; Sun, 4 Nov 2012 01:08:27 +0000 (UTC) Received: from localhost (babel2.noc.teithe.gr [195.251.240.240] (may be forged)) by alpha.it.teithe.gr (8.14.3/8.14.3/Debian-9.4) with ESMTP id qA418HiP032498 for ; Sun, 4 Nov 2012 03:08:17 +0200 Received: from dsl-aauy6k.dyn.edudsl.gr (dsl-aauy6k.dyn.edudsl.gr [37.32.186.140]) by webmail2.teithe.gr (Horde Framework) with HTTP; Sun, 04 Nov 2012 03:08:25 +0200 Message-ID: <20121104030825.18214e6ylkj7vi15@webmail2.teithe.gr> Date: Sun, 04 Nov 2012 03:08:25 +0200 From: Tzanetos Balitsaris To: freebsd-hackers@freebsd.org Subject: Subversion - Sync Branch with Trunk MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) H3 (4.3.9) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 01:08:28 -0000 Hello, During my GSoC project, I branched HEAD in order to use it for the development of the client side part of my project. After some changes, I tried to sync my branch with HEAD but I have faced an error. Now, I am trying once again to sync my branch with HEAD, but I get the exact same error. The error is the following: svn: E175002: PROPFIND of '/socsvn/!svn/bc/236241/mirror/FreeBSD/head/sys/dev/usb/controller': 207 Multi-Status (https://socsvn.freebsd.org) svn: E175002: Error reading spooled REPORT request response The error appears after 2 hours of syncing my branch with HEAD, with U (updated) as the svn status code for most of the files, and 4-6 of them that I resolved the conflicts by selecting tf (theirs-full). This is what I do to sync my branch with trunk (as described in the SVN Book [1]): # the root of my working copy, it contains the .svn, client-side, and server-side directories cd /home/tzabal/akcrs svn status if [ no local modifications reported ]; then svn merge https://socsvn.freebsd.org/socsvn/mirror/FreeBSD/head client-side/akcrs-head fi Can you propose any solutions in order to sync my branch with HEAD? Regards [1] Keeping a Branch in Sync, http://svnbook.red-bean.com/en/1.6/svn.branchmerge.basicmerging.html P.S. The project's code is located at http://svnweb.freebsd.org/socsvn/soc2012/tzabal/ -- Tzanetos Balitsaris ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 14:16:46 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0DD1FA23 for ; Sun, 4 Nov 2012 14:16:46 +0000 (UTC) (envelope-from heliocentric@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 87D138FC14 for ; Sun, 4 Nov 2012 14:16:44 +0000 (UTC) Received: by mail-bk0-f54.google.com with SMTP id jf20so1955814bkc.13 for ; Sun, 04 Nov 2012 06:16:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=ttD06boqS71E7uTQCsAS6v/wyAP3kpl4rTX0ZuA+m/A=; b=e8J9GT2dIkegiLUS4BJfkaVtGN9TgEJrw90QL8lR/wPGEEgdyFlm48iFmMQwUxUsfG Sp6uoTN9JiPq1GkeuuPRN15IizQsZaEaSCwihHOa4knW7GGqQvdaNxgAPWXDu16U8lwB 73AQVXSPWv+H6cIyyi6fYw5Nefj0zaYBqjHmk2hOifi1m7T6DSWxaJaMRbD8ax+w+aCn co7lFw/QQJ2QIMDHR7i7MgcrBAva7oTnr5RniP3QSm7jd17xdhg5z8x68FGTzNYEtWNG yWIOQPXLI/MfOp7PvqikHKSSB2xCcfrLctXCVqa1DFTkFZUarblNvCnKBatUlQ+dwSp5 t0Mg== MIME-Version: 1.0 Received: by 10.204.131.87 with SMTP id w23mr1675588bks.73.1352038603561; Sun, 04 Nov 2012 06:16:43 -0800 (PST) Sender: heliocentric@gmail.com Received: by 10.204.227.135 with HTTP; Sun, 4 Nov 2012 06:16:43 -0800 (PST) In-Reply-To: <1351973531.1120.118.camel@revolution.hippie.lan> References: <1351967919.1120.102.camel@revolution.hippie.lan> <50957793.8060709@delphij.net> <1351973531.1120.118.camel@revolution.hippie.lan> Date: Sun, 4 Nov 2012 09:16:43 -0500 X-Google-Sender-Auth: a3DRJB4l8tfv4fw3x5hOCsvgtNc Message-ID: Subject: Re: watchdogd, jemalloc, and mlockall From: Dylan Cochran To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 14:16:46 -0000 Have you already tried something like opt.lg_chunk? This, combined with other options for the library (man 3 jemalloc), should reduce the space from 8MB down to 16K, or so. (approximation, I'm being liberal for jemalloc's internal bookkeeping size). For a special case like watchdogd, this would make more sense in general (given it should be designed to do no allocations/deletions during normal operation anyway). For other programs, this would be as unwise as statically linking them. The 'perfect' solution would obviously be improving the library manager (rtld) to only mmap() function pages it needs, though I will admit I'm not sure if the ELF format is even capable of supporting something like that, what other problems it would cause down the road, or if it even attempts to do this already (I haven't looked at the runtime linker code since 7.0). By the way, remember that when you compare static v dynamic, that the runtime linker does allocate private memory to handle the resolving of symbols to virtual memory addresses. That may skew your memory usage figures a bit. On Sat, Nov 3, 2012 at 4:12 PM, Ian Lepore wrote: > > On Sat, 2012-11-03 at 12:59 -0700, Xin Li wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA256 > > > > On 11/3/12 11:38 AM, Ian Lepore wrote: > > > In an attempt to un-hijack the thread about memory usage increase > > > between 6.4 and 9.x, I'm starting a new thread here related to my > > > recent discovery that watchdogd uses a lot more memory since it > > > began using mlockall(2). > > > > > > I tried statically linking watchdogd and it made a small difference > > > in RSS, presumably because it doesn't wire down all of libc and > > > libm. > > > > Speaking for this, the last time I brought this up, someone (can't > > remember, I think it was phk@) argued that the shared library would > > use only one copy of memory, while statically linked ones would be > > duplicated and thus use more memory. I haven't yet tried to prove or > > challenge that, though. > > That sounds right to me... if 3 or 4 daemons were to eventually be > statically linked because of mlockall(), then each of them would have > its own private copy of strlen(), and malloc(), and so on; we'd be back > to the bad old days before shared libs came along. Each program would > contain its own copy of only the routines from the library that it uses, > not the entire library in each program. > > On the other hand, if even one daemon linked with shared libc uses > mlockall(), then all of libc gets wired. As I understand it, only one > physical copy of libc would exist in memory, still shared by almost all > running apps. The entire contents of the library would continuously > occupy physical memory, even the parts that no apps are using. > > It's hard to know how to weigh the various tradeoffs. I suspect there's > no one correct answer. > > -- Ian > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 14:37:32 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 35DAC405 for ; Sun, 4 Nov 2012 14:37:32 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (unknown [IPv6:2001:610:1108:5012::107]) by mx1.freebsd.org (Postfix) with ESMTP id B97A28FC08 for ; Sun, 4 Nov 2012 14:37:31 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 12C291203BF for ; Sun, 4 Nov 2012 15:37:28 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id DF89F2848C; Sun, 4 Nov 2012 15:37:27 +0100 (CET) Date: Sun, 4 Nov 2012 15:37:27 +0100 From: Jilles Tjoelker To: freebsd-hackers@FreeBSD.org Subject: [patch] rtld: fix fd leak with parallel dlopen and fork/exec Message-ID: <20121104143727.GB35520@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 14:37:32 -0000 Rtld does not set FD_CLOEXEC on its internal file descriptors; therefore, such a file descriptor may be passed to a process created by another thread running in parallel to dlopen() or fdlopen(). The race is easy to trigger with the below dlopen_exec_race.c that performs the two in parallel repeatedly, for example ./dlopen_exec_race /lib/libedit.so.7 | grep lib No other threads are expected to be running during parsing of the hints and libmap files but the file descriptors need not be passed to child processes so I added O_CLOEXEC there as well. The O_CLOEXEC flag is ignored by older kernels so it will not cause breakage when people try the unsupported action of running new rtld on old kernel. However, the fcntl(F_DUPFD_CLOEXEC) will fail with [EINVAL] on an old kernel so this patch will break fdlopen() with old kernels. I suppose this is OK because fdlopen() is not used in the vital parts of buildworld and the like. Index: libexec/rtld-elf/libmap.c =================================================================== --- libexec/rtld-elf/libmap.c (revision 240561) +++ libexec/rtld-elf/libmap.c (working copy) @@ -121,7 +121,7 @@ } } - fd = open(rpath, O_RDONLY); + fd = open(rpath, O_RDONLY | O_CLOEXEC); if (fd == -1) { dbg("lm_parse_file: open(\"%s\") failed, %s", rpath, rtld_strerror(errno)); Index: libexec/rtld-elf/rtld.c =================================================================== --- libexec/rtld-elf/rtld.c (revision 240561) +++ libexec/rtld-elf/rtld.c (working copy) @@ -1598,7 +1598,7 @@ /* Keep from trying again in case the hints file is bad. */ hints = ""; - if ((fd = open(ld_elf_hints_path, O_RDONLY)) == -1) + if ((fd = open(ld_elf_hints_path, O_RDONLY | O_CLOEXEC)) == -1) return (NULL); if (read(fd, &hdr, sizeof hdr) != sizeof hdr || hdr.magic != ELFHINTS_MAGIC || @@ -2046,13 +2046,13 @@ */ fd = -1; if (fd_u == -1) { - if ((fd = open(path, O_RDONLY)) == -1) { + if ((fd = open(path, O_RDONLY | O_CLOEXEC)) == -1) { _rtld_error("Cannot open \"%s\"", path); free(path); return (NULL); } } else { - fd = dup(fd_u); + fd = fcntl(fd_u, F_DUPFD_CLOEXEC, 0); if (fd == -1) { _rtld_error("Cannot dup fd"); free(path); dlopen_exec_race.c: #include #include #include #include #include #include #include #include extern char **environ; static void * dlopen_thread(void *arg) { const char *path = arg; void *handle; for (;;) { handle = dlopen(path, RTLD_LOCAL | RTLD_NOW); if (handle == NULL) continue; dlclose(handle); } return NULL; } static void filestat_loop(void) { int error, status; pid_t pid, wpid; for (;;) { error = posix_spawnp(&pid, "sh", NULL, NULL, (char *[]){ "sh", "-c", "procstat -f $$", NULL }, environ); if (error != 0) errc(1, error, "posix_spawnp"); do wpid = waitpid(pid, &status, 0); while (wpid == -1 && errno == EINTR); if (wpid == -1) err(1, "waitpid"); if (status != 0) errx(1, "sh -c failed"); } } int main(int argc, char *argv[]) { pthread_t td; int error; if (argc != 2) { fprintf(stderr, "Usage: %s dso\n", argv[0]); return 1; } error = pthread_create(&td, NULL, dlopen_thread, argv[1]); if (error != 0) errc(1, error, "pthread_create"); filestat_loop(); return 0; } fdlopen_exec_race.c: #include #include #include #include #include #include #include #include #include #include extern char **environ; static void * dlopen_thread(void *arg) { const char *path = arg; int fd; void *handle; for (;;) { fd = open(path, O_RDONLY | O_CLOEXEC); if (fd == -1) err(1, "open %s", path); handle = fdlopen(fd, RTLD_LOCAL | RTLD_NOW); close(fd); if (handle == NULL) continue; dlclose(handle); } return NULL; } static void filestat_loop(void) { int error, status; pid_t pid, wpid; for (;;) { error = posix_spawnp(&pid, "sh", NULL, NULL, (char *[]){ "sh", "-c", "procstat -f $$", NULL }, environ); if (error != 0) errc(1, error, "posix_spawnp"); do wpid = waitpid(pid, &status, 0); while (wpid == -1 && errno == EINTR); if (wpid == -1) err(1, "waitpid"); if (status != 0) errx(1, "sh -c failed"); } } int main(int argc, char *argv[]) { pthread_t td; int error; if (argc != 2) { fprintf(stderr, "Usage: %s dso\n", argv[0]); return 1; } error = pthread_create(&td, NULL, dlopen_thread, argv[1]); if (error != 0) errc(1, error, "pthread_create"); filestat_loop(); return 0; } -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 14:49:29 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8E340EA3 for ; Sun, 4 Nov 2012 14:49:29 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kostikbel-1-pt.tunnel.tserv11.ams1.ipv6.he.net [IPv6:2001:470:1f14:13d6::2]) by mx1.freebsd.org (Postfix) with ESMTP id EAF558FC17 for ; Sun, 4 Nov 2012 14:49:28 +0000 (UTC) Received: from tom.home (localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qA4EnO3t036943; Sun, 4 Nov 2012 16:49:24 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qA4EnOd3036942; Sun, 4 Nov 2012 16:49:24 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 4 Nov 2012 16:49:24 +0200 From: Konstantin Belousov To: Jilles Tjoelker Subject: Re: [patch] rtld: fix fd leak with parallel dlopen and fork/exec Message-ID: <20121104144924.GN73505@kib.kiev.ua> References: <20121104143727.GB35520@stack.nl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="p0HNO2YbtFeVXwJ3" Content-Disposition: inline In-Reply-To: <20121104143727.GB35520@stack.nl> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=0.2 required=5.0 tests=ALL_TRUSTED, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 14:49:29 -0000 --p0HNO2YbtFeVXwJ3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Nov 04, 2012 at 03:37:27PM +0100, Jilles Tjoelker wrote: > Rtld does not set FD_CLOEXEC on its internal file descriptors; > therefore, such a file descriptor may be passed to a process created by > another thread running in parallel to dlopen() or fdlopen(). >=20 > The race is easy to trigger with the below dlopen_exec_race.c that > performs the two in parallel repeatedly, for example > ./dlopen_exec_race /lib/libedit.so.7 | grep lib >=20 > No other threads are expected to be running during parsing of the hints > and libmap files but the file descriptors need not be passed to child > processes so I added O_CLOEXEC there as well. >=20 > The O_CLOEXEC flag is ignored by older kernels so it will not cause > breakage when people try the unsupported action of running new rtld on > old kernel. However, the fcntl(F_DUPFD_CLOEXEC) will fail with [EINVAL] > on an old kernel so this patch will break fdlopen() with old kernels. I > suppose this is OK because fdlopen() is not used in the vital parts of > buildworld and the like. The patch should be fine. We definitely do not support running newer binaries on older kernels, so the worries about the fdlopen(3) are not sustained. Please commit. >=20 > Index: libexec/rtld-elf/libmap.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- libexec/rtld-elf/libmap.c (revision 240561) > +++ libexec/rtld-elf/libmap.c (working copy) > @@ -121,7 +121,7 @@ > } > } > =20 > - fd =3D open(rpath, O_RDONLY); > + fd =3D open(rpath, O_RDONLY | O_CLOEXEC); > if (fd =3D=3D -1) { > dbg("lm_parse_file: open(\"%s\") failed, %s", rpath, > rtld_strerror(errno)); > Index: libexec/rtld-elf/rtld.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- libexec/rtld-elf/rtld.c (revision 240561) > +++ libexec/rtld-elf/rtld.c (working copy) > @@ -1598,7 +1598,7 @@ > /* Keep from trying again in case the hints file is bad. */ > hints =3D ""; > =20 > - if ((fd =3D open(ld_elf_hints_path, O_RDONLY)) =3D=3D -1) > + if ((fd =3D open(ld_elf_hints_path, O_RDONLY | O_CLOEXEC)) =3D=3D -1) > return (NULL); > if (read(fd, &hdr, sizeof hdr) !=3D sizeof hdr || > hdr.magic !=3D ELFHINTS_MAGIC || > @@ -2046,13 +2046,13 @@ > */ > fd =3D -1; > if (fd_u =3D=3D -1) { > - if ((fd =3D open(path, O_RDONLY)) =3D=3D -1) { > + if ((fd =3D open(path, O_RDONLY | O_CLOEXEC)) =3D=3D -1) { > _rtld_error("Cannot open \"%s\"", path); > free(path); > return (NULL); > } > } else { > - fd =3D dup(fd_u); > + fd =3D fcntl(fd_u, F_DUPFD_CLOEXEC, 0); > if (fd =3D=3D -1) { > _rtld_error("Cannot dup fd"); > free(path); >=20 >=20 > dlopen_exec_race.c: >=20 > #include > #include >=20 > #include > #include > #include > #include > #include > #include >=20 > extern char **environ; >=20 > static void * > dlopen_thread(void *arg) > { > const char *path =3D arg; > void *handle; >=20 > for (;;) { > handle =3D dlopen(path, RTLD_LOCAL | RTLD_NOW); > if (handle =3D=3D NULL) > continue; > dlclose(handle); > } > return NULL; > } >=20 > static void > filestat_loop(void) > { > int error, status; > pid_t pid, wpid; >=20 > for (;;) { > error =3D posix_spawnp(&pid, "sh", NULL, NULL, > (char *[]){ "sh", "-c", "procstat -f $$", NULL }, > environ); > if (error !=3D 0) > errc(1, error, "posix_spawnp"); > do > wpid =3D waitpid(pid, &status, 0); > while (wpid =3D=3D -1 && errno =3D=3D EINTR); > if (wpid =3D=3D -1) > err(1, "waitpid"); > if (status !=3D 0) > errx(1, "sh -c failed"); > } > } >=20 > int > main(int argc, char *argv[]) > { > pthread_t td; > int error; >=20 > if (argc !=3D 2) > { > fprintf(stderr, "Usage: %s dso\n", argv[0]); > return 1; > } >=20 > error =3D pthread_create(&td, NULL, dlopen_thread, argv[1]); > if (error !=3D 0) > errc(1, error, "pthread_create"); >=20 > filestat_loop(); >=20 > return 0; > } >=20 > fdlopen_exec_race.c: >=20 > #include > #include >=20 > #include > #include > #include > #include > #include > #include > #include > #include >=20 > extern char **environ; >=20 > static void * > dlopen_thread(void *arg) > { > const char *path =3D arg; > int fd; > void *handle; >=20 > for (;;) { > fd =3D open(path, O_RDONLY | O_CLOEXEC); > if (fd =3D=3D -1) > err(1, "open %s", path); > handle =3D fdlopen(fd, RTLD_LOCAL | RTLD_NOW); > close(fd); > if (handle =3D=3D NULL) > continue; > dlclose(handle); > } > return NULL; > } >=20 > static void > filestat_loop(void) > { > int error, status; > pid_t pid, wpid; >=20 > for (;;) { > error =3D posix_spawnp(&pid, "sh", NULL, NULL, > (char *[]){ "sh", "-c", "procstat -f $$", NULL }, > environ); > if (error !=3D 0) > errc(1, error, "posix_spawnp"); > do > wpid =3D waitpid(pid, &status, 0); > while (wpid =3D=3D -1 && errno =3D=3D EINTR); > if (wpid =3D=3D -1) > err(1, "waitpid"); > if (status !=3D 0) > errx(1, "sh -c failed"); > } > } >=20 > int > main(int argc, char *argv[]) > { > pthread_t td; > int error; >=20 > if (argc !=3D 2) > { > fprintf(stderr, "Usage: %s dso\n", argv[0]); > return 1; > } >=20 > error =3D pthread_create(&td, NULL, dlopen_thread, argv[1]); > if (error !=3D 0) > errc(1, error, "pthread_create"); >=20 > filestat_loop(); >=20 > return 0; > } >=20 > --=20 > Jilles Tjoelker > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --p0HNO2YbtFeVXwJ3 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlCWgHMACgkQC3+MBN1Mb4g24QCfVN8037vLPSuN26+SSURSo0jF 7usAoJQMVfGhVmD1qkO/J29bHMVGpBhv =to7n -----END PGP SIGNATURE----- --p0HNO2YbtFeVXwJ3-- From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 14:59:32 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2644F569; Sun, 4 Nov 2012 14:59:32 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id D10418FC08; Sun, 4 Nov 2012 14:59:31 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qA4ExODi072549; Sun, 4 Nov 2012 07:59:25 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qA4Ex1ec011219; Sun, 4 Nov 2012 07:59:01 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Subject: Re: watchdogd, jemalloc, and mlockall From: Ian Lepore To: Dylan Cochran In-Reply-To: References: <1351967919.1120.102.camel@revolution.hippie.lan> <50957793.8060709@delphij.net> <1351973531.1120.118.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Sun, 04 Nov 2012 07:59:01 -0700 Message-ID: <1352041141.1120.146.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 14:59:32 -0000 On Sun, 2012-11-04 at 09:16 -0500, Dylan Cochran wrote: > Have you already tried something like opt.lg_chunk? This, combined > with other options for the library (man 3 jemalloc), should reduce the > space from 8MB down to 16K, or so. (approximation, I'm being liberal > for jemalloc's internal bookkeeping size). For a special case like > watchdogd, this would make more sense in general (given it should be > designed to do no allocations/deletions during normal operation > anyway). For other programs, this would be as unwise as statically > linking them. > I had completely missed the fact that jemalloc had its own manpage, thank you. Given that new information I think the pieces are in place to put watchdogd on a memory diet. I'll work up a patch in the next couple days -- Ian > The 'perfect' solution would obviously be improving the library > manager (rtld) to only mmap() function pages it needs, though I will > admit I'm not sure if the ELF format is even capable of supporting > something like that, what other problems it would cause down the road, > or if it even attempts to do this already (I haven't looked at the > runtime linker code since 7.0). > > By the way, remember that when you compare static v dynamic, that the > runtime linker does allocate private memory to handle the resolving of > symbols to virtual memory addresses. That may skew your memory usage > figures a bit. > > On Sat, Nov 3, 2012 at 4:12 PM, Ian Lepore > wrote: > > > > On Sat, 2012-11-03 at 12:59 -0700, Xin Li wrote: > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA256 > > > > > > On 11/3/12 11:38 AM, Ian Lepore wrote: > > > > In an attempt to un-hijack the thread about memory usage increase > > > > between 6.4 and 9.x, I'm starting a new thread here related to my > > > > recent discovery that watchdogd uses a lot more memory since it > > > > began using mlockall(2). > > > > > > > > I tried statically linking watchdogd and it made a small difference > > > > in RSS, presumably because it doesn't wire down all of libc and > > > > libm. > > > > > > Speaking for this, the last time I brought this up, someone (can't > > > remember, I think it was phk@) argued that the shared library would > > > use only one copy of memory, while statically linked ones would be > > > duplicated and thus use more memory. I haven't yet tried to prove or > > > challenge that, though. > > > > That sounds right to me... if 3 or 4 daemons were to eventually be > > statically linked because of mlockall(), then each of them would have > > its own private copy of strlen(), and malloc(), and so on; we'd be back > > to the bad old days before shared libs came along. Each program would > > contain its own copy of only the routines from the library that it uses, > > not the entire library in each program. > > > > On the other hand, if even one daemon linked with shared libc uses > > mlockall(), then all of libc gets wired. As I understand it, only one > > physical copy of libc would exist in memory, still shared by almost all > > running apps. The entire contents of the library would continuously > > occupy physical memory, even the parts that no apps are using. > > > > It's hard to know how to weigh the various tradeoffs. I suspect there's > > no one correct answer. > > > > -- Ian > > > > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 16:36:17 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 11F6AD63 for ; Sun, 4 Nov 2012 16:36:17 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id B1ED08FC16 for ; Sun, 4 Nov 2012 16:36:16 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id k10so9090433iea.13 for ; Sun, 04 Nov 2012 08:36:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=63yDsjz8PAzKhlYSRmP1Yuke8U3kKrQTEyn7nqTVvKY=; b=GiKnnCf8VU7NN0soy6m7E8/VisiG7ofGGaoIrFFg8Qd3iIVDPBWF9ms7qwByJwVIjO HLBZNVWLqi5sSqGPMbv5CGizh8UbWiPKJcOrbq/hbdlmSLe1xXauQq0R6t5R7nz9uwDU 253v9QsI2lX66FBznsX6QjRNitRP3HMGXRgXssM/2Zbm+WnjJNatzZvXE+PN+3/CoGIN YS7b2wJEG6xzBFBis5X4BIlaIPco9r8siCbnOc6+jhQxmO1e4a3LMpHnflnFdOjqcgOy RxXsIwrf+sdedKLcCir5BdSR9xb2Glk7oNihJVdnU+E53vqWn989hESB8E8iCbCIWtvy 5j1g== Received: by 10.50.88.168 with SMTP id bh8mr7197806igb.71.1352046976152; Sun, 04 Nov 2012 08:36:16 -0800 (PST) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id l8sm4085807igo.13.2012.11.04.08.36.14 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 04 Nov 2012 08:36:15 -0800 (PST) Sender: Warner Losh Subject: Re: watchdogd, jemalloc, and mlockall Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1351968635.1120.110.camel@revolution.hippie.lan> Date: Sun, 4 Nov 2012 09:36:13 -0700 Content-Transfer-Encoding: 7bit Message-Id: <2D77347F-9913-4727-AA57-204AC266E15C@bsdimp.com> References: <1351967919.1120.102.camel@revolution.hippie.lan> <20121103184143.GC73505@kib.kiev.ua> <1351968635.1120.110.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQn0m7lLtYXcP4VL5KoxO8e8URW5Ug4Q07oTUmBPfrNWS9UTrt60GjaBV1XbcXM+NPsy97mt Cc: Konstantin Belousov , freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 16:36:17 -0000 On Nov 3, 2012, at 12:50 PM, Ian Lepore wrote: > On Sat, 2012-11-03 at 20:41 +0200, Konstantin Belousov wrote: >> On Sat, Nov 03, 2012 at 12:38:39PM -0600, Ian Lepore wrote: >>> In an attempt to un-hijack the thread about memory usage increase >>> between 6.4 and 9.x, I'm starting a new thread here related to my recent >>> discovery that watchdogd uses a lot more memory since it began using >>> mlockall(2). >>> >>> I tried statically linking watchdogd and it made a small difference in >>> RSS, presumably because it doesn't wire down all of libc and libm. >>> >>> VSZ RSS >>> 10236 10164 Dynamic >>> 8624 8636 Static >>> >>> Those numbers are from ps -u on an arm platform. I just updated the PR >>> (bin/173332) with some procstat -v output comparing with/without >>> mlockall(). >>> >>> It appears that the bulk of the new RSS bloat comes from jemalloc >>> allocating vmspace in 8MB chunks. With mlockall(MCL_FUTURE) in effect >>> that leads to wiring 8MB to satisfy what probably amounts to a few >>> hundred bytes of malloc'd memory. >>> >>> It would probably also be a good idea to remove the floating point from >>> watchdogd to avoid wiring all of libm. The floating point is used just >>> to turn the timeout-in-seconds into a power-of-two-nanoseconds value. >>> There's probably a reasonably efficient way to do that without calling >>> log(), considering that it only happens once at program startup. >> >> No, I propose to add a switch to turn on/off the mlockall() call. >> I have no opinion on the default value of the suggested switch. > > In a patch I submitted along with the PR, I added code to query the > vm.swap_enabled sysctl and only call mlockall() when swapping is > enabled. > > Nobody yet has said anything about what seems to me to be the real > problem here: jemalloc grabs 8MB at a time even if you only need to > malloc a few bytes, and there appears to be no way to control that > behavior. Or maybe there's a knob in there that didn't jump out at me > on a quick glance through the header files. Isn't that only for non-production builds? Warner From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 16:47:05 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5E96BF8A; Sun, 4 Nov 2012 16:47:05 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id C39668FC08; Sun, 4 Nov 2012 16:47:04 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qA4Gl3Cc075619; Sun, 4 Nov 2012 09:47:03 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qA4GkeLN011274; Sun, 4 Nov 2012 09:46:40 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Subject: Re: watchdogd, jemalloc, and mlockall From: Ian Lepore To: Warner Losh In-Reply-To: <2D77347F-9913-4727-AA57-204AC266E15C@bsdimp.com> References: <1351967919.1120.102.camel@revolution.hippie.lan> <20121103184143.GC73505@kib.kiev.ua> <1351968635.1120.110.camel@revolution.hippie.lan> <2D77347F-9913-4727-AA57-204AC266E15C@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Sun, 04 Nov 2012 09:46:40 -0700 Message-ID: <1352047600.1120.148.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 16:47:05 -0000 On Sun, 2012-11-04 at 09:36 -0700, Warner Losh wrote: > On Nov 3, 2012, at 12:50 PM, Ian Lepore wrote: > > > On Sat, 2012-11-03 at 20:41 +0200, Konstantin Belousov wrote: > >> On Sat, Nov 03, 2012 at 12:38:39PM -0600, Ian Lepore wrote: > >>> In an attempt to un-hijack the thread about memory usage increase > >>> between 6.4 and 9.x, I'm starting a new thread here related to my recent > >>> discovery that watchdogd uses a lot more memory since it began using > >>> mlockall(2). > >>> > >>> I tried statically linking watchdogd and it made a small difference in > >>> RSS, presumably because it doesn't wire down all of libc and libm. > >>> > >>> VSZ RSS > >>> 10236 10164 Dynamic > >>> 8624 8636 Static > >>> > >>> Those numbers are from ps -u on an arm platform. I just updated the PR > >>> (bin/173332) with some procstat -v output comparing with/without > >>> mlockall(). > >>> > >>> It appears that the bulk of the new RSS bloat comes from jemalloc > >>> allocating vmspace in 8MB chunks. With mlockall(MCL_FUTURE) in effect > >>> that leads to wiring 8MB to satisfy what probably amounts to a few > >>> hundred bytes of malloc'd memory. > >>> > >>> It would probably also be a good idea to remove the floating point from > >>> watchdogd to avoid wiring all of libm. The floating point is used just > >>> to turn the timeout-in-seconds into a power-of-two-nanoseconds value. > >>> There's probably a reasonably efficient way to do that without calling > >>> log(), considering that it only happens once at program startup. > >> > >> No, I propose to add a switch to turn on/off the mlockall() call. > >> I have no opinion on the default value of the suggested switch. > > > > In a patch I submitted along with the PR, I added code to query the > > vm.swap_enabled sysctl and only call mlockall() when swapping is > > enabled. > > > > Nobody yet has said anything about what seems to me to be the real > > problem here: jemalloc grabs 8MB at a time even if you only need to > > malloc a few bytes, and there appears to be no way to control that > > behavior. Or maybe there's a knob in there that didn't jump out at me > > on a quick glance through the header files. > > Isn't that only for non-production builds? > > Warner I don't think so, I discovered this on my tflex unit running -current, and it's built with MALLOC_PRODUCTION defined because it doesn't have enough ram to boot without it defined. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 17:02:10 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1187F57F; Sun, 4 Nov 2012 17:02:10 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id C17FB8FC15; Sun, 4 Nov 2012 17:02:09 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qA4H23mV076114; Sun, 4 Nov 2012 10:02:03 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qA4H1e9W011293; Sun, 4 Nov 2012 10:01:40 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Subject: Re: watchdogd, jemalloc, and mlockall From: Ian Lepore To: Warner Losh In-Reply-To: <2D77347F-9913-4727-AA57-204AC266E15C@bsdimp.com> References: <1351967919.1120.102.camel@revolution.hippie.lan> <20121103184143.GC73505@kib.kiev.ua> <1351968635.1120.110.camel@revolution.hippie.lan> <2D77347F-9913-4727-AA57-204AC266E15C@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Sun, 04 Nov 2012 10:01:40 -0700 Message-ID: <1352048500.1120.151.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 17:02:10 -0000 On Sun, 2012-11-04 at 09:36 -0700, Warner Losh wrote: > On Nov 3, 2012, at 12:50 PM, Ian Lepore wrote: > > > On Sat, 2012-11-03 at 20:41 +0200, Konstantin Belousov wrote: > >> On Sat, Nov 03, 2012 at 12:38:39PM -0600, Ian Lepore wrote: > >>> In an attempt to un-hijack the thread about memory usage increase > >>> between 6.4 and 9.x, I'm starting a new thread here related to my recent > >>> discovery that watchdogd uses a lot more memory since it began using > >>> mlockall(2). > >>> > >>> I tried statically linking watchdogd and it made a small difference in > >>> RSS, presumably because it doesn't wire down all of libc and libm. > >>> > >>> VSZ RSS > >>> 10236 10164 Dynamic > >>> 8624 8636 Static > >>> > >>> Those numbers are from ps -u on an arm platform. I just updated the PR > >>> (bin/173332) with some procstat -v output comparing with/without > >>> mlockall(). > >>> > >>> It appears that the bulk of the new RSS bloat comes from jemalloc > >>> allocating vmspace in 8MB chunks. With mlockall(MCL_FUTURE) in effect > >>> that leads to wiring 8MB to satisfy what probably amounts to a few > >>> hundred bytes of malloc'd memory. > >>> > >>> It would probably also be a good idea to remove the floating point from > >>> watchdogd to avoid wiring all of libm. The floating point is used just > >>> to turn the timeout-in-seconds into a power-of-two-nanoseconds value. > >>> There's probably a reasonably efficient way to do that without calling > >>> log(), considering that it only happens once at program startup. > >> > >> No, I propose to add a switch to turn on/off the mlockall() call. > >> I have no opinion on the default value of the suggested switch. > > > > In a patch I submitted along with the PR, I added code to query the > > vm.swap_enabled sysctl and only call mlockall() when swapping is > > enabled. > > > > Nobody yet has said anything about what seems to me to be the real > > problem here: jemalloc grabs 8MB at a time even if you only need to > > malloc a few bytes, and there appears to be no way to control that > > behavior. Or maybe there's a knob in there that didn't jump out at me > > on a quick glance through the header files. > > Isn't that only for non-production builds? > > Warner I just realized the implication of what you asked. I think it must be that jemalloc always allocates big chunks of vmspace at a time (unless tuned to do otherwise; I haven't looked into the tuning stuff yet), but when MALLOC_PRODUCTION isn't defined it also touches all the pages within that allocated space, presumably to lay in known byte patterns or other debugging info. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 19:53:05 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 786D1458 for ; Sun, 4 Nov 2012 19:53:05 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 05B068FC15 for ; Sun, 4 Nov 2012 19:53:04 +0000 (UTC) Received: (qmail 42186 invoked from network); 4 Nov 2012 21:28:51 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 4 Nov 2012 21:28:51 -0000 Message-ID: <5096C79E.6020400@freebsd.org> Date: Sun, 04 Nov 2012 20:53:02 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: John Baldwin Subject: Re: kernel module parallel build? References: <5083D84E.50903@freebsd.org> <201210220928.52778.jhb@freebsd.org> In-Reply-To: <201210220928.52778.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 19:53:05 -0000 On 22.10.2012 15:28, John Baldwin wrote: > On Sunday, October 21, 2012 7:11:10 am Andre Oppermann wrote: >> What's keeping kernel modules from building in parallel with >> "make -j8"? > > They don't for you? They do for me either via 'make buildkernel' > or the old method. They do, but only partially. Within a module the files are built in parallel. However the module directories seem to be serialized. I'm a Makefile noob though. -- Andre From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 5 18:52:15 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BAEE020E for ; Mon, 5 Nov 2012 18:52:15 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell0.rawbw.com (shell0.rawbw.com [198.144.192.45]) by mx1.freebsd.org (Postfix) with ESMTP id 874708FC08 for ; Mon, 5 Nov 2012 18:52:15 +0000 (UTC) Received: from eagle.yuri.org (stunnel@localhost [127.0.0.1]) (authenticated bits=0) by shell0.rawbw.com (8.14.4/8.14.4) with ESMTP id qA5IqDmS002841 for ; Mon, 5 Nov 2012 10:52:14 -0800 (PST) (envelope-from yuri@rawbw.com) Message-ID: <50980ADD.4010402@rawbw.com> Date: Mon, 05 Nov 2012 10:52:13 -0800 From: Yuri User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:16.0) Gecko/20121023 Thunderbird/16.0.1 MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org Subject: pgbench performance is lagging compared to Linux and DragonflyBSD? Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2012 18:52:15 -0000 There is the post by DragonflyBSD folks that claims that Linux and DragonflyBSD are quite ahead of FreeBSD on pgbench test on 12 Core 2x Xeon X5650 with 24 threads. Here are their results with graphs: http://lists.dragonflybsd.org/pipermail/users/attachments/20121010/7996ff88/attachment-0002.pdf And here is their original post: http://lists.dragonflybsd.org/pipermail/users/2012-October/017536.html I am not sure if this is the problem of some sysctl or kernel parameters or some serious system issue. It looks like the DragonflyBSD folks made a goal to do well on pgbench and got to the level of ~88% of linux with 80 clients. Yuri From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 5 20:52:16 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CBCBA63D for ; Mon, 5 Nov 2012 20:52:16 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4B9708FC12 for ; Mon, 5 Nov 2012 20:52:15 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id b5so5670174lbd.13 for ; Mon, 05 Nov 2012 12:52:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sboeQbIuShItXj0hwce2SnZfwjPNxgr5ighX9r3/5AA=; b=HivQ/LGgSXF1JRbpVk5dvQAcBrn+nzTxxUv4FS/eZDYbSmOAVz/v1kBszhQOhAgYUg f5hB1sjFhlvxsiis1jrTQRiYheqfvHz5HowB0rMREp9KYrtHWABK1/D0ugJwqU0gompb Ant+3So6J/PRvSr/qn3qzG89B5pTOYmDJ4lNxa9hOw7nGWaPXLDDYxoba8rfJjnkLa7i nJrllSKua9UylNfF0FZ8zY2avK9g1CWBvFrJgAo+/lN/MhbBRqZYnm5/4pgSC3bh5tCy lLLaoeKFRwwkPMizRNBYnQqlcFfE5Huq8HEgHKlVjECJh8E3KRqbsZZzLtj2DgTwf0yZ +zJg== MIME-Version: 1.0 Received: by 10.152.106.212 with SMTP id gw20mr10326286lab.8.1352148734702; Mon, 05 Nov 2012 12:52:14 -0800 (PST) Received: by 10.112.144.101 with HTTP; Mon, 5 Nov 2012 12:52:14 -0800 (PST) In-Reply-To: <50980ADD.4010402@rawbw.com> References: <50980ADD.4010402@rawbw.com> Date: Mon, 5 Nov 2012 12:52:14 -0800 Message-ID: Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? From: Garrett Cooper To: Yuri Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2012 20:52:16 -0000 On Mon, Nov 5, 2012 at 10:52 AM, Yuri wrote: > There is the post by DragonflyBSD folks that claims that Linux and > DragonflyBSD are quite ahead of FreeBSD on pgbench test on 12 Core 2x Xeon > X5650 with 24 threads. > Here are their results with graphs: http://lists.dragonflybsd.org/** > pipermail/users/attachments/**20121010/7996ff88/attachment-**0002.pdf > And here is their original post: http://lists.dragonflybsd.org/** > pipermail/users/2012-October/**017536.html > > I am not sure if this is the problem of some sysctl or kernel parameters > or some serious system issue. > > It looks like the DragonflyBSD folks made a goal to do well on pgbench and > got to the level of ~88% of linux with 80 clients. The important item that has been left out (or is just implied as OS level defaults) is sysctl/tunable variables set in the *BSD OSes (on DFly, FreeBSD, and NetBSD). Unfortunately (based on my experience) FreeBSD could be a lot better when it comes to defaults, and more tuning is required to get better performance. So if they're working with the OS defaults, this might not be a fair equivalent to the best performance that FreeBSD can yield, but it's probably fair to do this for the sake of repeatability and to prove what these OSes can do out of the box. This is in addition to the [lock] contention issues that jeffr@ and a few others are working on alleviating. FWIW, I think that the last time scheduler benchmarks from anyone at @FreeBSD.org (was kris@ the last one, or has flo@ run benchmarks since then? My Googling is a bit inconclusive) was run was several years ago as well, so if Linux has improved I'm not at all surprised. However, please also take into consideration that the hardware then and the hardware now are grossly different. So the interactions between the hardware then and the hardware now might differ greatly. In short, more inspection needs to be done to figure out whether or not the findings are true [with caveats] or false. Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 5 21:30:35 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 41396751 for ; Mon, 5 Nov 2012 21:30:35 +0000 (UTC) (envelope-from sfourman@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id C42138FC17 for ; Mon, 5 Nov 2012 21:30:34 +0000 (UTC) Received: by mail-ee0-f54.google.com with SMTP id c50so4086372eek.13 for ; Mon, 05 Nov 2012 13:30:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=EYDcu3Ph5Hgz/+Myx0FocGqI7AV/BfiOcRordIFsmkQ=; b=DFnkv5939pHOoBxaQQAgSWfKq/clDjtUhy1+IoEkC0ltKaw44Ec6zJp6urvpZFQ6GP wpt2jjuQG5VAbIaxNvlfg7N7JQz45GK1W/4JsoQzleLUqJhEbssnmAZPPKM2mI4IEI6b DpFgWvQL0s4w8KkntzrnKDhYXeZhMAdlDrShnRbVv0XgEGPTxHQip6K0o1Qi2nnosXfZ s+DDoPrOi9jTKSkJziRq3aDticPgSw/FHz8AbmTP733p/TTdlzC9hFtSYv9K0Z7GZFSs TVquRted8DIpiT2XSNzrNZqBV0ULOzCHhkchtWL1GNbRO2TZFalm7XvWr1E+4V7096Yv jDuA== MIME-Version: 1.0 Received: by 10.14.172.137 with SMTP id t9mr41334012eel.2.1352151033495; Mon, 05 Nov 2012 13:30:33 -0800 (PST) Received: by 10.14.140.79 with HTTP; Mon, 5 Nov 2012 13:30:33 -0800 (PST) In-Reply-To: References: <50980ADD.4010402@rawbw.com> Date: Mon, 5 Nov 2012 15:30:33 -0600 Message-ID: Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? From: "Sam Fourman Jr." To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: Yuri , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2012 21:30:35 -0000 > The important item that has been left out (or is just implied as OS level > defaults) is sysctl/tunable variables set in the *BSD OSes (on DFly, > FreeBSD, and NetBSD). Unfortunately (based on my experience) FreeBSD could > be a lot better when it comes to defaults, and more tuning is required to > get better performance. So if they're working with the OS defaults, this > might not be a fair equivalent to the best performance that FreeBSD can > yield, but it's probably fair to do this for the sake of repeatability and > to prove what these OSes can do out of the box. This is in addition to the > [lock] contention issues that jeffr@ and a few others are working on > alleviating. > has someone wrote a howto for how to tune pgsql 9+ in FreeBSD? im mostly asking here to get information posted here for future reference, as google will pick this thread up and it will help others. -- Sam Fourman Jr. From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 5 23:43:53 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6B23D530 for ; Mon, 5 Nov 2012 23:43:53 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 221B98FC12 for ; Mon, 5 Nov 2012 23:43:53 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qA5NhkfF028520 for ; Mon, 5 Nov 2012 16:43:52 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qA5NhO7B012693 for ; Mon, 5 Nov 2012 16:43:24 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Subject: procstat -v question From: Ian Lepore To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="us-ascii" Date: Mon, 05 Nov 2012 16:43:23 -0700 Message-ID: <1352159003.1120.200.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2012 23:43:53 -0000 In a line of procstat -v output such as this: PID START END PRT RES PRES REF SHD FL TP PATH 60065 0x200c1000 0x201c3000 r-x 182 0 17 8 CN vn /usr/lib/libstdc++.so.6 Does that 182 resident pages mean that the process being displayed is referencing that many pages itself, or does that represent how many pages are resident due to all the references from all the processes that have the library open? -- Ian From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 09:26:23 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 43DDC68C for ; Tue, 6 Nov 2012 09:26:23 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 9DF4E8FC08 for ; Tue, 6 Nov 2012 09:26:21 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA69QGge018247; Tue, 6 Nov 2012 10:26:16 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA69QGnO018244; Tue, 6 Nov 2012 10:26:16 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Tue, 6 Nov 2012 10:26:16 +0100 (CET) From: Wojciech Puchar To: Garrett Cooper Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: Message-ID: References: <50980ADD.4010402@rawbw.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Tue, 06 Nov 2012 10:26:16 +0100 (CET) Cc: Yuri , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 09:26:23 -0000 > defaults) is sysctl/tunable variables set in the *BSD OSes (on DFly, > FreeBSD, and NetBSD). Unfortunately (based on my experience) FreeBSD could > be a lot better when it comes to defaults, and more tuning is required to actually FreeBSD defaults are actually good for COMMON usage. and can be tuned. default MAXBSIZE is one exception. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 09:28:31 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 82ACB7BA for ; Tue, 6 Nov 2012 09:28:31 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id D64F18FC0C for ; Tue, 6 Nov 2012 09:28:30 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA69SLOR018253; Tue, 6 Nov 2012 10:28:21 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA69SLi4018250; Tue, 6 Nov 2012 10:28:21 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Tue, 6 Nov 2012 10:28:21 +0100 (CET) From: Wojciech Puchar To: Ian Lepore Subject: Re: procstat -v question In-Reply-To: <1352159003.1120.200.camel@revolution.hippie.lan> Message-ID: References: <1352159003.1120.200.camel@revolution.hippie.lan> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Tue, 06 Nov 2012 10:28:21 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 09:28:31 -0000 > > Does that 182 resident pages mean that the process being displayed is > referencing that many pages itself, or does that represent how many to THIS process. Long time ago when i asked about multiple programs mapping same large files - i learned that pagetables are always per process. When you run your process then you get pagefault for every page (page group) you reference first. Just no disk I/O when such page is already in memory because other process already needed it. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 09:31:46 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 163FA8DE for ; Tue, 6 Nov 2012 09:31:46 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 5FD548FC18 for ; Tue, 6 Nov 2012 09:31:45 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA69P6XZ018241; Tue, 6 Nov 2012 10:25:06 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA69P5if018238; Tue, 6 Nov 2012 10:25:05 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Tue, 6 Nov 2012 10:25:05 +0100 (CET) From: Wojciech Puchar To: Yuri Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: <50980ADD.4010402@rawbw.com> Message-ID: References: <50980ADD.4010402@rawbw.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Tue, 06 Nov 2012 10:25:06 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 09:31:46 -0000 > some serious system issue. > > It looks like the DragonflyBSD folks made a goal to do well on pgbench and > got to the level of ~88% of linux with 80 clients. It's just bad that anyone judge and (even worse) modify/tune operating system to do well in SINGLE benchmark running basically single program doing few repetitive things. Linux is tuned to win in benchmark and it does, while having disastrous performance in normal unix style usage - multiple different programs doing multiple different things for multiple different users - in the same time. This is a case with at least 99% of users. The less than 1% that have so heavy load that needs separete machine dedicated to single program doing one thing - could use linux (if it REALLY will be better in production workload ) or even better - use some dedicated hardware just for this, if it exist. Does machine that is dedicated to run single program need OS at all? In such "benchmark" FreeBSD with UFS wins hands down and that's the reason i use it. Still it is interesting WHY FreeBSD is slower in that special case, and if improvements on general behaviour can be found then it's nice to do them. I tried dragonflybsd some time ago and it's performance on normal usage is disastrous. Seems like Matthew Dillion years after splitting from FreeBSD because "the algorithms used in FreeBSD were plain wrong" - cannot do this better but still waste time and still at all cost want to prove he can. Tuning operating system for single benchmark is an example of that childish behaviour. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 11:02:07 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9149DF88; Tue, 6 Nov 2012 11:02:07 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1B8838FC08; Tue, 6 Nov 2012 11:02:05 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id b5so358576lbd.13 for ; Tue, 06 Nov 2012 03:02:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=w4yGp9U/n96Elsl/O7xSXBb9MXYxxBlOVjOlABXhw6c=; b=aXpAb2kvJDbRv9cxBgs7i6NdKYg+hjIphfhQebQItTwBW6oqfNJ1j1tjKrA0pnmF4c dyM4w6b8Mw5wn9LZ7m0GNRo8j4Bq95vYzAhk70RYky6gZKus7ygZDSsdPGWyMnC8HqlK OYFGeeHaBriAJDpCmMzTfAjTXiNIv84oFNOYtK0URdw3VJAjMWOwmKcmp6Yfnq/5aaNO 8WCFJayw0Vr8WuSz06vZ1j2LKI5Blw7HNOjKqEDmDakAPfCSsKkTpU4YdlohH65LZOYz sYqT2deeuDcQK09Y72j3cbZIyNALGjFikDOsJ+G2CCuY3GCKzW0An7GbpDUZJIlkG7CX zmIA== MIME-Version: 1.0 Received: by 10.152.123.103 with SMTP id lz7mr667341lab.21.1352199724609; Tue, 06 Nov 2012 03:02:04 -0800 (PST) Sender: asmrookie@gmail.com Received: by 10.112.30.37 with HTTP; Tue, 6 Nov 2012 03:02:04 -0800 (PST) In-Reply-To: References: <50587F8D.9060102@FreeBSD.org> <5058C68B.1010508@FreeBSD.org> <50596019.5060708@FreeBSD.org> <505AF836.7050004@FreeBSD.org> <506C461F.2050008@FreeBSD.org> Date: Tue, 6 Nov 2012 11:02:04 +0000 X-Google-Sender-Auth: 5HYjbN29SWFpmKacqw44Yeol1UI Message-ID: Subject: Re: ule+smp: small optimization for turnstile priority lending From: Attilio Rao To: Andriy Gapon Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers , Jeff Roberson X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: attilio@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 11:02:07 -0000 On 10/29/12, Attilio Rao wrote: > On Mon, Oct 29, 2012 at 4:56 PM, Attilio Rao wrote: >> On Wed, Oct 3, 2012 at 3:05 PM, Andriy Gapon wrote: >>> on 20/09/2012 16:14 Attilio Rao said the following: >>>> On 9/20/12, Andriy Gapon wrote: >>> [snip] >>>>> The patch works well as far as I can tell. Thank you! >>>>> There is one warning with full witness enables but it appears to be >>>>> harmless >>>>> (so >>>>> far): >>>> >>>> Andriy, >>>> thanks a lot for your testing and reports you made so far. >>>> Unfortunately I'm going off for 2 weeks now and I won't work on >>>> FreeBSD for that timeframe. I will get back to those in 2 weeks then. >>>> If you want to play more with this idea feel free to extend/fix/etc. >>>> this patch. >>> >>> Unfortunately I haven't found time to work on this further, but I have >>> some >>> additional thoughts. >>> >>> First, I think that the witness message was not benign and it actually >>> warns about >>> the same kind of deadlock that I originally had. >>> The problem is that sched_lend_prio() is called with target thread's >>> td_lock held, >>> which is a lock of tdq on the thread's CPU. Then, in your patch, we >>> acquire >>> current tdq's lock to modify its load. But if two tdq locks are to be >>> held at the >>> same time, then they must be locked using tdq_lock_pair, so that lock >>> order is >>> maintained. With the patch no tdq lock order can be maintained (can >>> arbitrary) >>> and thus it is possible to run into a deadlock. >> >> Indeed. I realized this after thinking about this problem while I was >> on holiday. >> >>> >>> I see two possibilities here, but don't rule out that there can be more. >>> >>> 1. Do something like my patch does. That is, manipulate current thread's >>> tdq in >>> advance before any other thread/tdq locks are acquired (or td_lock is >>> changed to >>> point to a different lock and current tdq is unlocked). The API can be >>> made more >>> generic in nature. E.g. it can look like this: >>> void >>> sched_thread_block(struct thread *td, int inhibitor) >>> { >>> struct tdq *tdq; >>> >>> THREAD_LOCK_ASSERT(td, MA_OWNED); >>> KASSERT(td == curthread, >>> ("sched_thread_block: only curthread is supported")); >>> tdq = TDQ_SELF(); >>> TDQ_LOCK_ASSERT(tdq, MA_OWNED); >>> MPASS(td->td_lock == TDQ_LOCKPTR(tdq)); >>> TD_SET_INHIB(td, inhibitor); >>> tdq_load_rem(tdq, td); >>> tdq_setlowpri(tdq, td); >>> } >>> >>> >>> 2. Try to do things from sched_lend_prio based on curthread's state. >>> This, as it >>> seems, would require completely lock-less manipulations of current tdq. >>> E.g. for >>> tdq_load we could use atomic operations (it is already accessed >>> locklessly, but >>> not modified). But for tdq_lowpri a more elaborate trick would be >>> required like >>> having a separate field for a temporary value. >> >> No, this is not going to work because tdq_lowpri and tdq_load are >> updated and used in the same critical path (and possibly also other >> tdq* members in tdq_runqueue_add(), for example, but I didn't bother >> to also check that). > > Ok, so here I wasn't completely right because td_lowpri and tdq_load > are not read in the same critical path (cpu_search() and > sched_pickcpu() in the self case). > However, I was over-estimating a factor in this patch: I don't think > we need to fix real tdq_load for self in that case before cpu_search() > because self is handled in a very special way, with its own > comparison. I then think that a patch local to sched_pickcpu() on the > self path should be enough. > > This brings us to another bigger issue, however. tdq_lowpri handling > is not perfect in that patch. Think about the case where the thread to > be switched out is, infact, the lowest priority one. In that case, > what happens is that what we should do is recalculating tdq_lowpri > which is usually a very expensive operation and requires TDQ self > locking to be in place to be brought on correctly. I think that the easy fix for this is to implement a second field called tdq_sec_lowpri where we store the next thread to be scheduled, after tdq_lowpri. This is going to not really change anything because we will get it for free when we already calculate tdq_lowpri and it will let us the implement a definitive logic that helps in the case you describe. However I still want to think about the base idea behind the current algorithm in the self/likely cpu pickup. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 11:03:50 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 26F3C462; Tue, 6 Nov 2012 11:03:50 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0EC008FC15; Tue, 6 Nov 2012 11:03:48 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id e12so290538lag.13 for ; Tue, 06 Nov 2012 03:03:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=+LBGboGTOeSATcHxlVekjrZH2kqM3UYoP6xRZgBMSs8=; b=YLSBhA6a3zvosLpWQ7Xav2Jy+KdUnyQ5MfIjAB8nmzhX0X87U7cr62uWs81z9WOwF8 Kpv2a9YzUOoQmkWk/fOaygH8mrOGMgODudvqunbZwYRnM+PyPmbi7qZA3MlJICrzXppj mbZ8PUVsw0y9Eg3pCj3i27NzaiSjgoROxJuUFm8XC62Lya301YzXNGW6nJ2s+vgnOP42 Uq4s2CzRBa+H6Z3NZwL12Fjq+1VgRtR7S5nyo8WRhq3rumH8jn5U+a/2uHJVprxFQvkS PVBUGDLG1/Klui7bCjckaQH1yGs23wpqJOxeosAbcMUPZaaDHELPhZ75aC3T6mzXuQ+t LkXw== MIME-Version: 1.0 Received: by 10.152.110.234 with SMTP id id10mr695449lab.15.1352199827535; Tue, 06 Nov 2012 03:03:47 -0800 (PST) Sender: asmrookie@gmail.com Received: by 10.112.30.37 with HTTP; Tue, 6 Nov 2012 03:03:47 -0800 (PST) In-Reply-To: <505AD2A5.6060008@freebsd.org> References: <50587F8D.9060102@FreeBSD.org> <505AD2A5.6060008@freebsd.org> Date: Tue, 6 Nov 2012 11:03:47 +0000 X-Google-Sender-Auth: Jj3FXjTSU3cpZsvwXWqRBfoTfmI Message-ID: Subject: Re: ule+smp: small optimization for turnstile priority lending From: Attilio Rao To: David Xu Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers , Jeff Roberson , Andriy Gapon X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: attilio@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 11:03:50 -0000 On 9/20/12, David Xu wrote: > On 2012/09/18 22:05, Andriy Gapon wrote: >> >> Here is a snippet that demonstrates the issue on a supposedly fully >> loaded >> 2-processor system: >> >> 136794 0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid >> 102818", >> state:"running", attributes: prio:122 >> >> 136793 0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid >> 111916", >> state:"yielding", attributes: prio:183, wmesg:"(null)", lockname:"(null)" >> >> 136792 1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 tid >> 100004", >> state:"running", attributes: prio:255 >> >> 136791 1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load", >> counter:0, >> attributes: none >> >> 136790 1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid >> 113473", >> state:"blocked", attributes: prio:122, wmesg:"(null)", lockname:"unp_mtx" >> >> 136789 1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load", >> counter:2, >> attributes: none >> >> 136788 1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid >> 113473", >> point:"wokeup", attributes: linkedto:"Xorg tid 102818" >> >> 136787 1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid >> 102818", >> state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473" >> >> 136786 1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load", >> counter:1, >> attributes: none >> >> 136785 1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid >> 102818", >> state:"runq rem", attributes: prio:176 >> >> 136784 1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid >> 102818", >> point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox tid >> 113473" >> >> 136783 1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid >> 113473", >> state:"running", attributes: prio:104 >> >> See how how the Xorg thread was forced from CPU 1 to CPU 0 where it >> preempted >> cc1plus thread (I do have preemption enabled) only to leave CPU 1 with >> zero load. >> >> Here is a proposed solution: >> >> turnstile_wait: optimize priority lending to a thread on a runqueue >> >> As the current thread is definitely going into mi_switch, it now >> removes >> its load before doing priority propagation which can potentially >> result >> in sched_add. In the SMP && ULE case the latter searches for the >> least loaded CPU to place a boosted thread, which is supposedly >> about >> to run. >> >> diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c >> index 8e466cd..3299cae 100644 >> --- a/sys/kern/sched_ule.c >> +++ b/sys/kern/sched_ule.c >> @@ -1878,7 +1878,10 @@ sched_switch(struct thread *td, struct thread >> *newtd, int >> flags) >> /* This thread must be going to sleep. */ >> TDQ_LOCK(tdq); >> mtx = thread_lock_block(td); >> - tdq_load_rem(tdq, td); >> +#if defined(SMP) >> + if ((flags & SW_TYPE_MASK) != SWT_TURNSTILE) >> +#endif >> + tdq_load_rem(tdq, td); >> } >> /* >> * We enter here with the thread blocked and assigned to the >> @@ -2412,6 +2415,21 @@ sched_rem(struct thread *td) >> tdq_setlowpri(tdq, NULL); >> } >> >> +void >> +sched_load_rem(struct thread *td) >> +{ >> + struct tdq *tdq; >> + >> + KASSERT(td == curthread, >> + ("sched_rem_load: only curthread is supported")); >> + KASSERT(td->td_oncpu == td->td_sched->ts_cpu, >> + ("thread running on cpu different from ts_cpu")); >> + tdq = TDQ_CPU(td->td_sched->ts_cpu); >> + TDQ_LOCK_ASSERT(tdq, MA_OWNED); >> + MPASS(td->td_lock == TDQ_LOCKPTR(tdq)); >> + tdq_load_rem(tdq, td); >> +} >> + >> /* >> * Fetch cpu utilization information. Updates on demand. >> */ >> diff --git a/sys/kern/subr_turnstile.c b/sys/kern/subr_turnstile.c >> index 31d16fe..d1d68e9 100644 >> --- a/sys/kern/subr_turnstile.c >> +++ b/sys/kern/subr_turnstile.c >> @@ -731,6 +731,13 @@ turnstile_wait(struct turnstile *ts, struct thread >> *owner, >> int queue) >> LIST_INSERT_HEAD(&ts->ts_free, td->td_turnstile, ts_hash); >> } >> thread_lock(td); >> +#if defined(SCHED_ULE) && defined(SMP) >> + /* >> + * Remove load earlier so that it does not affect cpu selection >> + * for a thread waken up due to priority lending, if any. >> + */ >> + sched_load_rem(td); >> +#endif >> thread_lock_set(td, &ts->ts_lock); >> td->td_turnstile = NULL; >> >> diff --git a/sys/sys/sched.h b/sys/sys/sched.h >> index 4b8387c..b1ead1b 100644 >> --- a/sys/sys/sched.h >> +++ b/sys/sys/sched.h >> @@ -110,6 +110,9 @@ void sched_preempt(struct thread *td); >> void sched_add(struct thread *td, int flags); >> void sched_clock(struct thread *td); >> void sched_rem(struct thread *td); >> +#if defined(SCHED_ULE) && defined(SMP) >> +void sched_load_rem(struct thread *td); >> +#endif >> void sched_tick(int cnt); >> void sched_relinquish(struct thread *td); >> struct thread *sched_choose(void); >> > > I found another scenario in taskqueue, in the function > taskqueue_terminate, current thread tries to wake > another thread up and sleep immediately, the tq_mutex sometimes > is a spinlock. So if you remove one thread load from current cpu > before wakeup, the resumed thread may be put on same cpu, > so it will optimize the cpu scheduling too. I think that in order to fit with sched_add() modifies I have in mind (see other patches within this thread) wakeup() should grow a new argument, or maybe we can use wakeup_flags() new KPI. If the latter is the case, I would also propose to let wakeup_one() to be absorbed into wakeup_flags() with its own flag. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 16:51:01 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D1C214F6 for ; Tue, 6 Nov 2012 16:51:01 +0000 (UTC) (envelope-from olivier@gid0.org) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 50C248FC15 for ; Tue, 6 Nov 2012 16:51:00 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id e12so663542lag.13 for ; Tue, 06 Nov 2012 08:50:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=wpGgFsm9cUY0TpTCsTDekn6f6TIu+v/masJECmq4Qz0=; b=gEebUVno5s/C4YYjLiHdRQJw5XbjtLGDYzYt66bngkxtKx8RCYVeaiPFRWsk35eWM7 UO88VfErKCL/GUl5Z3yh0kJ+R4ZoA4Zs2Pa6kH5LUTCMRTwh9leNHs28b4Hkebe3z6DG Bs8kyJfRhiVbzNmtI5XS5FrAWy3o6mRHo4iVv7FcVZz4bhB9jI+LA+bWtAzAFFUnNbPc LAiV/utiA2+phXqLAmAndAWiuru9DdYnPlUn8zo8JtP1vDHfiFT1M3GXYejkgFwjGJL7 2prdjDSSPYSekqIIG/mlTfbXtjNARjVXbxdiI2cNBiQHDmi0NEowATEciGq366abv8Av Xs3w== MIME-Version: 1.0 Received: by 10.152.106.79 with SMTP id gs15mr1683056lab.31.1352220659679; Tue, 06 Nov 2012 08:50:59 -0800 (PST) Received: by 10.112.135.137 with HTTP; Tue, 6 Nov 2012 08:50:59 -0800 (PST) In-Reply-To: References: <50980ADD.4010402@rawbw.com> Date: Tue, 6 Nov 2012 17:50:59 +0100 Message-ID: Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? From: Olivier Smedts To: Wojciech Puchar Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQltL2dfn9A+gsHkbgz63kLPZpC1249UeZErN1MK8sNIBXEdW0vNd4JMY5p3gB573213WTKK Cc: Yuri , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 16:51:01 -0000 2012/11/6 Wojciech Puchar : > Tuning operating system for single benchmark is an example of that childish > behaviour. LOL. That's what "we" did several years ago : http://people.freebsd.org/~kris/scaling/dfly.html I won't blame dflybsd for benchmarking something specific. Maybe there is something we can learn from what they did for the 3.2 release. -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier@gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 16:52:01 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3F7B1706 for ; Tue, 6 Nov 2012 16:52:01 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 30AC48FC08 for ; Tue, 6 Nov 2012 16:51:59 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA6GpmoO020331; Tue, 6 Nov 2012 17:51:48 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA6Gpms4020328; Tue, 6 Nov 2012 17:51:48 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Tue, 6 Nov 2012 17:51:48 +0100 (CET) From: Wojciech Puchar To: "Samuel J. Greear" Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: Message-ID: References: <50980ADD.4010402@rawbw.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Tue, 06 Nov 2012 17:51:49 +0100 (CET) Cc: Yuri , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 16:52:01 -0000 > Your entire email is conjecture, the performance of DragonFly 3.2 is > improved across the board vs 3.0. Not just batch performance, > interactive performance (especially under X11) is also greatly > improved. i must try. i checked 3.0 and earlier versions and it was a disaster. Relative to FreeBSD of course. > > Sam > > On Tue, Nov 6, 2012 at 2:25 AM, Wojciech Puchar > wrote: >>> some serious system issue. >>> >>> It looks like the DragonflyBSD folks made a goal to do well on pgbench and >>> got to the level of ~88% of linux with 80 clients. >> >> >> It's just bad that anyone judge and (even worse) modify/tune operating >> system to do well in SINGLE benchmark running basically single program doing >> few repetitive things. >> >> Linux is tuned to win in benchmark and it does, while having disastrous >> performance in normal unix style usage - multiple different programs doing >> multiple different things for multiple different users - in the same time. >> >> This is a case with at least 99% of users. The less than 1% that have so >> heavy load that needs separete machine dedicated to single program doing one >> thing - could use linux (if it REALLY will be better in production workload >> ) or even better - use some dedicated hardware just for this, if it exist. >> >> Does machine that is dedicated to run single program need OS at all? >> >> >> In such "benchmark" FreeBSD with UFS wins hands down and that's the reason i >> use it. >> >> >> Still it is interesting WHY FreeBSD is slower in that special case, and if >> improvements on general behaviour can be found then it's nice to do them. >> >> >> I tried dragonflybsd some time ago and it's performance on normal usage is >> disastrous. Seems like Matthew Dillion years after splitting from FreeBSD >> because "the algorithms used in FreeBSD were plain wrong" - cannot do this >> better but still waste time and still at all cost want to prove he can. >> >> Tuning operating system for single benchmark is an example of that childish >> behaviour. >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 16:55:34 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5233D813 for ; Tue, 6 Nov 2012 16:55:34 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 8C8718FC14 for ; Tue, 6 Nov 2012 16:55:33 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA6GtV3N020371; Tue, 6 Nov 2012 17:55:31 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA6GtV0K020368; Tue, 6 Nov 2012 17:55:31 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Tue, 6 Nov 2012 17:55:31 +0100 (CET) From: Wojciech Puchar To: Olivier Smedts Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: Message-ID: References: <50980ADD.4010402@rawbw.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Tue, 06 Nov 2012 17:55:32 +0100 (CET) Cc: Yuri , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 16:55:34 -0000 >> Tuning operating system for single benchmark is an example of that childish >> behaviour. > > LOL. That's what "we" did several years ago : > http://people.freebsd.org/~kris/scaling/dfly.html i've seen that page some time ago but i don't really care of it. i just wasn't interested. Still - DOING such benchmark is good, as it can show general problems in used algorithms. But working on software to make it better in some kind of synthetic benchmark is common in commercial software world. ("We have more performance per buck than company X") From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 16:44:10 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D6B59271 for ; Tue, 6 Nov 2012 16:44:10 +0000 (UTC) (envelope-from sjg@evilcode.net) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 971A68FC1B for ; Tue, 6 Nov 2012 16:44:10 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id k10so1193044iea.13 for ; Tue, 06 Nov 2012 08:44:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=YhOY1Pe/i8WVNVXOyHvaxfyfLLKxFlIDFrCrZ2INIMI=; b=GUr0gy0WRMg8h3GpQBnaO5M/FW5wEW/KWVlgvPMC7qn/i8t2dbsFQNBnRySq3LHdRt DfzzdLHEQcFibdagrHQeaEHCB5bwqEbxrLSBxHeRq7uJCxWSFkAkIET7q22jdY8xSt6p CcL9+mHbj5eC+avXjzJqrQQroKUA1AzP7trYIy4g9qfxl+RzmbgQhEuUCL2BkOL0v83Z DYeRgWvXPCLm5FQzxiQXwJZ13Yrj+YBZZ76YiI0jLOE5Hj8swZCOhPfu6jVDYmKNT1jE LhzJzo57EySAYFdtOOWydisSSASXPheH6IOIEt7IBjqTQM5l7HiOakM/r1qrFLyA4xky 2keg== MIME-Version: 1.0 Received: by 10.50.95.161 with SMTP id dl1mr13885563igb.0.1352220249919; Tue, 06 Nov 2012 08:44:09 -0800 (PST) Received: by 10.64.29.3 with HTTP; Tue, 6 Nov 2012 08:44:09 -0800 (PST) In-Reply-To: References: <50980ADD.4010402@rawbw.com> Date: Tue, 6 Nov 2012 09:44:09 -0700 Message-ID: Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? From: "Samuel J. Greear" To: Wojciech Puchar Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmaVghgCCMyegbvJsjzdZE8rVTa1Iv3k3NZck6j/P2ZZmiTuTv6KFKE3VZukHJsJbqJwe2e X-Mailman-Approved-At: Tue, 06 Nov 2012 17:06:50 +0000 Cc: Yuri , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 16:44:11 -0000 Your entire email is conjecture, the performance of DragonFly 3.2 is improved across the board vs 3.0. Not just batch performance, interactive performance (especially under X11) is also greatly improved. Sam On Tue, Nov 6, 2012 at 2:25 AM, Wojciech Puchar wrote: >> some serious system issue. >> >> It looks like the DragonflyBSD folks made a goal to do well on pgbench and >> got to the level of ~88% of linux with 80 clients. > > > It's just bad that anyone judge and (even worse) modify/tune operating > system to do well in SINGLE benchmark running basically single program doing > few repetitive things. > > Linux is tuned to win in benchmark and it does, while having disastrous > performance in normal unix style usage - multiple different programs doing > multiple different things for multiple different users - in the same time. > > This is a case with at least 99% of users. The less than 1% that have so > heavy load that needs separete machine dedicated to single program doing one > thing - could use linux (if it REALLY will be better in production workload > ) or even better - use some dedicated hardware just for this, if it exist. > > Does machine that is dedicated to run single program need OS at all? > > > In such "benchmark" FreeBSD with UFS wins hands down and that's the reason i > use it. > > > Still it is interesting WHY FreeBSD is slower in that special case, and if > improvements on general behaviour can be found then it's nice to do them. > > > I tried dragonflybsd some time ago and it's performance on normal usage is > disastrous. Seems like Matthew Dillion years after splitting from FreeBSD > because "the algorithms used in FreeBSD were plain wrong" - cannot do this > better but still waste time and still at all cost want to prove he can. > > Tuning operating system for single benchmark is an example of that childish > behaviour. > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 17:46:54 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CCD3E24E for ; Tue, 6 Nov 2012 17:46:54 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id 94F688FC08 for ; Tue, 6 Nov 2012 17:46:54 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bi1so547503pad.13 for ; Tue, 06 Nov 2012 09:46:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:x-mailer:from:subject:date :to; bh=jWoa5gkzLAaGUURvOMQS9YOUmMw9nj8Qk+0B06rRL3k=; b=mh6w21Nk+K3nXiBWzsz012Knaa7e9orSCLpamGfOmk1ph+g8mqogOMyrCBEtdZ3Gug NR4q8slJWQdmzIg9h9QZ28ro6LHbHSovTVy9uCHWEUOOvXHGb73IykFQ3dbEkWNiOxeg OaJgro4ecsudt2txhmU1zCqzt23bXVAg/GwNITpszqA7QzPYrULUvATWZDxbTVRiD9Sx aQ1I0mBNZ49Ex3wDdltKJ/1oq5nsohStApazDofJxQ9SSIDm4AXTOaF4OtyB/7kRhcae LMYqzbzlzGmpYn5zm2fkC/itW2Ra/Qg5LLIuVJiOV0hmo1eixLLIIdN2zcHu9LalTr6R YhRQ== Received: by 10.68.224.161 with SMTP id rd1mr5600088pbc.49.1352224013186; Tue, 06 Nov 2012 09:46:53 -0800 (PST) Received: from [10.77.14.93] (mobile-166-147-081-002.mycingular.net. [166.147.81.2]) by mx.google.com with ESMTPS id j4sm12773490pax.31.2012.11.06.09.46.40 (version=SSLv3 cipher=OTHER); Tue, 06 Nov 2012 09:46:52 -0800 (PST) References: <50980ADD.4010402@rawbw.com> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (10A403) From: Garrett Cooper Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? Date: Tue, 6 Nov 2012 09:46:22 -0800 To: Wojciech Puchar Cc: Olivier Smedts , Yuri , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 17:46:54 -0000 On Nov 6, 2012, at 8:55 AM, Wojciech Puchar = wrote: >>> Tuning operating system for single benchmark is an example of that child= ish >>> behaviour. >>=20 >> LOL. That's what "we" did several years ago : >> http://people.freebsd.org/~kris/scaling/dfly.html >=20 > i've seen that page some time ago but i don't really care of it. > i just wasn't interested. >=20 > Still - DOING such benchmark is good, as it can show general problems in u= sed algorithms. >=20 > But working on software to make it better in some kind of synthetic benchm= ark is common in commercial software world. ("We have more performance per b= uck than company X") "Synthetic benchmarks" as you put it shouldn't be the ultimate basis for a d= ecision, but instead allow users to gauge whether or not a certain software o= r hardware configuration is suitable for their given workload. No more, no l= ess. The fact that they're being used in this manner is a bit like a salesma= n selling snake oil as the results aren't necessarily the result of a "best"= configuration for all competing platforms, but instead an unknown configura= tion in this case. A similar statement about the importance of micro benchmarks can be made... Thanks, -Garrett= From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 17:54:47 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4DA57448 for ; Tue, 6 Nov 2012 17:54:47 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-da0-f54.google.com (mail-da0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 174DF8FC14 for ; Tue, 6 Nov 2012 17:54:46 +0000 (UTC) Received: by mail-da0-f54.google.com with SMTP id z9so321282dad.13 for ; Tue, 06 Nov 2012 09:54:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:x-mailer:from:subject:date :to; bh=MXV0bYAkLrWH+vn5tKB9cu45Q20ENUuKGJh3p7eeJqQ=; b=IqAzUj/N5tlRchgXamnScfqXo6TU6s2zZuo3gIwbEsVCuCaatM+bm4tUq2a0vfK2+u CDyYlu4oljaOWZ9nHvm3AMu0daxt/P1D2p60Zf0ohpEH8FqIHeEO2QBxYVyTcqv+a0Js 8N/SHtEEFII3kQkQipFgZ97xKnutx04L7S+FR59UESR+ge7up/iPfFo5J7/FEOz60Rdb D6iRsPIia36GvypAEdeYfqrBhiemaU+ogpe55FExyPmjf/m2aP6nC4DpE/8puqBw6chc m8Mu+e405S10AW6sYFp/BtXu5vFxyowhVco/1F3DT0EjcZlkhyKtq73PohyH/ExIcC8q k7yA== Received: by 10.68.225.34 with SMTP id rh2mr5542392pbc.78.1352224486530; Tue, 06 Nov 2012 09:54:46 -0800 (PST) Received: from [10.77.14.93] (mobile-166-147-081-002.mycingular.net. [166.147.81.2]) by mx.google.com with ESMTPS id pv7sm12668250pbc.20.2012.11.06.09.54.44 (version=SSLv3 cipher=OTHER); Tue, 06 Nov 2012 09:54:45 -0800 (PST) References: <50980ADD.4010402@rawbw.com> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (10A403) From: Garrett Cooper Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? Date: Tue, 6 Nov 2012 09:54:39 -0800 To: Wojciech Puchar Cc: Yuri , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 17:54:47 -0000 On Nov 6, 2012, at 1:26 AM, Wojciech Puchar = wrote: >> defaults) is sysctl/tunable variables set in the *BSD OSes (on DFly, >> FreeBSD, and NetBSD). Unfortunately (based on my experience) FreeBSD coul= d >> be a lot better when it comes to defaults, and more tuning is required to= >=20 > actually FreeBSD defaults are actually good for COMMON usage. and can be t= uned. >=20 > default MAXBSIZE is one exception. "Common usage" is vague. While FreeBSD might do ok for some applications (de= v box, simple workstation/laptop, etc), there are other areas that require a= dditional tuning to get better perf that arguably shouldn't as much (or ther= e should be templates for doing so): 10GbE and mbuf and network tuning; file= server and file descriptor, network tuning, etc; low latency desktop and sc= heduler tweaking; etc. Not to say that freebsd is entirely at fault, but because it's more of a com= modity OS that Linux, more tweaking is required... Thanks, -Garrett= From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 17:59:03 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 84B62630 for ; Tue, 6 Nov 2012 17:59:03 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell0.rawbw.com (shell0.rawbw.com [198.144.192.45]) by mx1.freebsd.org (Postfix) with ESMTP id 66C418FC0A for ; Tue, 6 Nov 2012 17:59:03 +0000 (UTC) Received: from eagle.yuri.org (stunnel@localhost [127.0.0.1]) (authenticated bits=0) by shell0.rawbw.com (8.14.4/8.14.4) with ESMTP id qA6Hwunp083183 for ; Tue, 6 Nov 2012 09:58:57 -0800 (PST) (envelope-from yuri@rawbw.com) Message-ID: <50994FE0.2070205@rawbw.com> Date: Tue, 06 Nov 2012 09:58:56 -0800 From: Yuri User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:16.0) Gecko/20121023 Thunderbird/16.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? References: <50980ADD.4010402@rawbw.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 17:59:03 -0000 On 11/05/2012 12:52, Garrett Cooper wrote: > FWIW, I think that the last time scheduler benchmarks from anyone at > @FreeBSD.org (was kris@ the last one, or has flo@ run benchmarks since I myself ran the similar test on i7 920 (4 cores 8 threads) @ 2.67 24GB with 9.1-RC3 with all the same params except shmem size was 4GB, not 6GB: http://i.imgur.com/mfnqr.png In DragonflyBSD tests FreeBSD peaked at 96k tps. And my machine, with roughly 3X lesser power, peaked at 44.5k tps. So in my test BSD performed relatively better. And graph shape is more resembling linux/DragonflyBSD ones. It looks like in their test FreeBSD behaved in somewhat impaired way. Any ideas what can I try to tune? Yuri From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 18:28:03 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EC743C9B for ; Tue, 6 Nov 2012 18:28:03 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from gw01.mail.saunalahti.fi (gw01.mail.saunalahti.fi [195.197.172.115]) by mx1.freebsd.org (Postfix) with ESMTP id 6AE5B8FC0C for ; Tue, 6 Nov 2012 18:28:03 +0000 (UTC) Received: from a91-153-116-96.elisa-laajakaista.fi (a91-153-116-96.elisa-laajakaista.fi [91.153.116.96]) by gw01.mail.saunalahti.fi (Postfix) with SMTP id 0DC94151617; Tue, 6 Nov 2012 20:27:53 +0200 (EET) Date: Tue, 6 Nov 2012 20:27:52 +0200 From: Jaakko Heinonen To: freebsd-hackers@FreeBSD.org Subject: [patch] md(4) and preloaded memory disks Message-ID: <20121106182752.GB2234@a91-153-116-96.elisa-laajakaista.fi> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: filip.palian@pjwstk.edu.pl X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 18:28:04 -0000 Hi, I plan to commit the patch below to disallow attaching preloaded memory disks via ioctl. I didn't find anything that would really use this undocumented feature. --- Disallow attaching preloaded memory disks via ioctl. - The feature is dangerous because the kernel code doesn't check validity of the memory address provided from user space. - It seems that mdconfig(8) never really supported attaching preloaded memory disks. - Preloaded memory disks are automatically attached during md(4) initialization. Thus there shouldn't be much use for the feature. PR: kern/169683 %%% Index: sbin/mdconfig/mdconfig.c =================================================================== --- sbin/mdconfig/mdconfig.c (revision 242608) +++ sbin/mdconfig/mdconfig.c (working copy) @@ -84,7 +84,7 @@ usage(void) " mdconfig -r -u unit -s size [-o [no]force]\n" " mdconfig -l [-v] [-n] [-u unit]\n" " mdconfig file\n"); - fprintf(stderr, "\t\ttype = {malloc, preload, vnode, swap}\n"); + fprintf(stderr, "\t\ttype = {malloc, vnode, swap}\n"); fprintf(stderr, "\t\toption = {cluster, compress, reserve}\n"); fprintf(stderr, "\t\tsize = %%d (512 byte blocks), %%db (B),\n"); fprintf(stderr, "\t\t %%dk (kB), %%dm (MB), %%dg (GB) or\n"); @@ -148,8 +148,6 @@ main(int argc, char **argv) if (!strcmp(optarg, "malloc")) { mdio.md_type = MD_MALLOC; mdio.md_options |= MD_AUTOUNIT | MD_COMPRESS; - } else if (!strcmp(optarg, "preload")) { - mdio.md_type = MD_PRELOAD; } else if (!strcmp(optarg, "vnode")) { mdio.md_type = MD_VNODE; mdio.md_options |= MD_CLUSTER | MD_AUTOUNIT | MD_COMPRESS; Index: sys/dev/md/md.c =================================================================== --- sys/dev/md/md.c (revision 242608) +++ sys/dev/md/md.c (working copy) @@ -845,15 +845,20 @@ mdinit(struct md_s *sc) DEVSTAT_ALL_SUPPORTED, DEVSTAT_TYPE_DIRECT, DEVSTAT_PRIORITY_MAX); } -/* - * XXX: we should check that the range they feed us is mapped. - * XXX: we should implement read-only. - */ - static int mdcreate_preload(struct md_s *sc, struct md_ioctl *mdio) { + /* + * Currently we disallow attaching preloaded memory disks via ioctl. + * Preloaded memory disks are automatically attached in mdinit(). + */ +#if 0 + /* + * XXX: We should check that the range they feed us is mapped and + * a valid memory disk. + * XXX: We should implement read-only. + */ if (mdio->md_options & ~(MD_AUTOUNIT | MD_FORCE)) return (EINVAL); if (mdio->md_base == 0) @@ -863,6 +868,9 @@ mdcreate_preload(struct md_s *sc, struct sc->pl_ptr = (u_char *)(uintptr_t)mdio->md_base; sc->pl_len = (size_t)sc->mediasize; return (0); +#else + return (EOPNOTSUPP); +#endif } %%% -- Jaakko From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 19:10:03 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A19DB838 for ; Tue, 6 Nov 2012 19:10:03 +0000 (UTC) (envelope-from sjg@evilcode.net) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5C8828FC0C for ; Tue, 6 Nov 2012 19:10:02 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id k10so1500776iea.13 for ; Tue, 06 Nov 2012 11:10:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=qkFYMTC1FAQjuDak5T1OVyS6vMptvxopRJFZFz0ur9c=; b=GVnrZDiyvdtI8RciaRR7wetuVGjYgt2jdMzzUk0aQOkhXueFsduC24kk3sQIocnlaC wemhrbQXMFeKYnTrpGJAbOj2SGbbWMkLuh3h7vupeerRgJY1ZuJZQ3T+Ijc+lrJMOOuL IPcbW0lWm0/ufp/BlW9lp7vthaNOlJ2Mc0cKJKr3yXkK1BLWOHwvMZC3Zyj+8oMbtJwl MQNf3SdefuqgASGHv3lVpmUeKZ6QJ+IZ9qju8TJ/DByOaeHDH3f37dJqOLl08z4hxO3D 91iyxzOfBJZLWPfpj+697EFCD+GNJ3ZDP523u82h+pVxdxD46puOeExlr48GA4cGLZST DCtQ== MIME-Version: 1.0 Received: by 10.50.213.34 with SMTP id np2mr2161179igc.57.1352229002527; Tue, 06 Nov 2012 11:10:02 -0800 (PST) Received: by 10.64.29.3 with HTTP; Tue, 6 Nov 2012 11:10:02 -0800 (PST) In-Reply-To: <50994FE0.2070205@rawbw.com> References: <50980ADD.4010402@rawbw.com> <50994FE0.2070205@rawbw.com> Date: Tue, 6 Nov 2012 12:10:02 -0700 Message-ID: Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? From: "Samuel J. Greear" To: Yuri Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQkJaKpevgRlZqFdjLCqYUAQyFNBzLA2YO0bMUR/HTFwZeloQuwTPrdU9KGEaKZ2a3qR2fd3 X-Mailman-Approved-At: Tue, 06 Nov 2012 19:37:45 +0000 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 19:10:03 -0000 On Tue, Nov 6, 2012 at 10:58 AM, Yuri wrote: > On 11/05/2012 12:52, Garrett Cooper wrote: >> >> FWIW, I think that the last time scheduler benchmarks from anyone at >> @FreeBSD.org (was kris@ the last one, or has flo@ run benchmarks since > > > I myself ran the similar test on i7 920 (4 cores 8 threads) @ 2.67 24GB with > 9.1-RC3 with all the same params except shmem size was 4GB, not 6GB: > http://i.imgur.com/mfnqr.png > In DragonflyBSD tests FreeBSD peaked at 96k tps. And my machine, with > roughly 3X lesser power, peaked at 44.5k tps. So in my test BSD performed > relatively better. And graph shape is more resembling linux/DragonflyBSD > ones. It looks like in their test FreeBSD behaved in somewhat impaired way. > > Any ideas what can I try to tune? > Single and multi-socket hardware are not really directly comparable in PostgreSQL tests. Sam > Yuri > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 19:40:07 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D8BCB4E6; Tue, 6 Nov 2012 19:40:07 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 926B78FC14; Tue, 6 Nov 2012 19:40:07 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 3F0D48A3FC; Tue, 6 Nov 2012 19:40:06 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id qA6Je5bE012863; Tue, 6 Nov 2012 19:40:05 GMT (envelope-from phk@phk.freebsd.dk) To: Jaakko Heinonen Subject: Re: [patch] md(4) and preloaded memory disks In-reply-to: <20121106182752.GB2234@a91-153-116-96.elisa-laajakaista.fi> From: "Poul-Henning Kamp" References: <20121106182752.GB2234@a91-153-116-96.elisa-laajakaista.fi> Date: Tue, 06 Nov 2012 19:40:05 +0000 Message-ID: <12862.1352230805@critter.freebsd.dk> X-Mailman-Approved-At: Tue, 06 Nov 2012 19:44:54 +0000 Cc: freebsd-hackers@FreeBSD.org, filip.palian@pjwstk.edu.pl X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 19:40:07 -0000 -------- In message <20121106182752.GB2234@a91-153-116-96.elisa-laajakaista.fi>, Jaakko Heinonen writes: >I plan to commit the patch below to disallow attaching preloaded memory >disks via ioctl. I didn't find anything that would really use this >undocumented feature. If I remember right, this was for the case where you had preloaded multiple images, and the kernel would only auto-discover the first one. All things (such as the disapperance of floppy disks), considered this feature should be removed. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 6 21:01:46 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5103E785 for ; Tue, 6 Nov 2012 21:01:46 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell0.rawbw.com (shell0.rawbw.com [198.144.192.45]) by mx1.freebsd.org (Postfix) with ESMTP id 1C4EC8FC0A for ; Tue, 6 Nov 2012 21:01:45 +0000 (UTC) Received: from eagle.yuri.org (stunnel@localhost [127.0.0.1]) (authenticated bits=0) by shell0.rawbw.com (8.14.4/8.14.4) with ESMTP id qA6L1iuX035102; Tue, 6 Nov 2012 13:01:44 -0800 (PST) (envelope-from yuri@rawbw.com) Message-ID: <50997AB8.40607@rawbw.com> Date: Tue, 06 Nov 2012 13:01:44 -0800 From: Yuri User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:16.0) Gecko/20121023 Thunderbird/16.0.1 MIME-Version: 1.0 To: "Samuel J. Greear" Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? References: <50980ADD.4010402@rawbw.com> <50994FE0.2070205@rawbw.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2012 21:01:46 -0000 On 11/06/2012 11:10, Samuel J. Greear wrote: > Single and multi-socket hardware are not really directly comparable in > PostgreSQL tests. So if the CPUs are split between sockets, would such system generally perform better or worse with PostgeSQL vs. non-split situation? Yuri From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 02:21:56 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0C765911; Wed, 7 Nov 2012 02:21:56 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id DE34F8FC08; Wed, 7 Nov 2012 02:21:55 +0000 (UTC) Received: from xyf.my.dom (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id qA72Lsu5095011; Wed, 7 Nov 2012 02:21:55 GMT (envelope-from davidxu@freebsd.org) Message-ID: <5099C5C9.4040703@freebsd.org> Date: Wed, 07 Nov 2012 10:22:01 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:14.0) Gecko/20120822 Thunderbird/14.0 MIME-Version: 1.0 To: attilio@freebsd.org Subject: Re: ule+smp: small optimization for turnstile priority lending References: <50587F8D.9060102@FreeBSD.org> <505AD2A5.6060008@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers , Jeff Roberson , Andriy Gapon X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 02:21:56 -0000 On 2012/11/06 19:03, Attilio Rao wrote: > On 9/20/12, David Xu wrote: >> On 2012/09/18 22:05, Andriy Gapon wrote: >>> >>> Here is a snippet that demonstrates the issue on a supposedly fully >>> loaded >>> 2-processor system: >>> >>> 136794 0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid >>> 102818", >>> state:"running", attributes: prio:122 >>> >>> 136793 0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid >>> 111916", >>> state:"yielding", attributes: prio:183, wmesg:"(null)", lockname:"(null)" >>> >>> 136792 1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 tid >>> 100004", >>> state:"running", attributes: prio:255 >>> >>> 136791 1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load", >>> counter:0, >>> attributes: none >>> >>> 136790 1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid >>> 113473", >>> state:"blocked", attributes: prio:122, wmesg:"(null)", lockname:"unp_mtx" >>> >>> 136789 1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load", >>> counter:2, >>> attributes: none >>> >>> 136788 1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid >>> 113473", >>> point:"wokeup", attributes: linkedto:"Xorg tid 102818" >>> >>> 136787 1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid >>> 102818", >>> state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473" >>> >>> 136786 1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load", >>> counter:1, >>> attributes: none >>> >>> 136785 1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid >>> 102818", >>> state:"runq rem", attributes: prio:176 >>> >>> 136784 1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid >>> 102818", >>> point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox tid >>> 113473" >>> >>> 136783 1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid >>> 113473", >>> state:"running", attributes: prio:104 >>> >>> See how how the Xorg thread was forced from CPU 1 to CPU 0 where it >>> preempted >>> cc1plus thread (I do have preemption enabled) only to leave CPU 1 with >>> zero load. >>> >>> Here is a proposed solution: >>> >>> turnstile_wait: optimize priority lending to a thread on a runqueue >>> >>> As the current thread is definitely going into mi_switch, it now >>> removes >>> its load before doing priority propagation which can potentially >>> result >>> in sched_add. In the SMP && ULE case the latter searches for the >>> least loaded CPU to place a boosted thread, which is supposedly >>> about >>> to run. >>> >>> diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c >>> index 8e466cd..3299cae 100644 >>> --- a/sys/kern/sched_ule.c >>> +++ b/sys/kern/sched_ule.c >>> @@ -1878,7 +1878,10 @@ sched_switch(struct thread *td, struct thread >>> *newtd, int >>> flags) >>> /* This thread must be going to sleep. */ >>> TDQ_LOCK(tdq); >>> mtx = thread_lock_block(td); >>> - tdq_load_rem(tdq, td); >>> +#if defined(SMP) >>> + if ((flags & SW_TYPE_MASK) != SWT_TURNSTILE) >>> +#endif >>> + tdq_load_rem(tdq, td); >>> } >>> /* >>> * We enter here with the thread blocked and assigned to the >>> @@ -2412,6 +2415,21 @@ sched_rem(struct thread *td) >>> tdq_setlowpri(tdq, NULL); >>> } >>> >>> +void >>> +sched_load_rem(struct thread *td) >>> +{ >>> + struct tdq *tdq; >>> + >>> + KASSERT(td == curthread, >>> + ("sched_rem_load: only curthread is supported")); >>> + KASSERT(td->td_oncpu == td->td_sched->ts_cpu, >>> + ("thread running on cpu different from ts_cpu")); >>> + tdq = TDQ_CPU(td->td_sched->ts_cpu); >>> + TDQ_LOCK_ASSERT(tdq, MA_OWNED); >>> + MPASS(td->td_lock == TDQ_LOCKPTR(tdq)); >>> + tdq_load_rem(tdq, td); >>> +} >>> + >>> /* >>> * Fetch cpu utilization information. Updates on demand. >>> */ >>> diff --git a/sys/kern/subr_turnstile.c b/sys/kern/subr_turnstile.c >>> index 31d16fe..d1d68e9 100644 >>> --- a/sys/kern/subr_turnstile.c >>> +++ b/sys/kern/subr_turnstile.c >>> @@ -731,6 +731,13 @@ turnstile_wait(struct turnstile *ts, struct thread >>> *owner, >>> int queue) >>> LIST_INSERT_HEAD(&ts->ts_free, td->td_turnstile, ts_hash); >>> } >>> thread_lock(td); >>> +#if defined(SCHED_ULE) && defined(SMP) >>> + /* >>> + * Remove load earlier so that it does not affect cpu selection >>> + * for a thread waken up due to priority lending, if any. >>> + */ >>> + sched_load_rem(td); >>> +#endif >>> thread_lock_set(td, &ts->ts_lock); >>> td->td_turnstile = NULL; >>> >>> diff --git a/sys/sys/sched.h b/sys/sys/sched.h >>> index 4b8387c..b1ead1b 100644 >>> --- a/sys/sys/sched.h >>> +++ b/sys/sys/sched.h >>> @@ -110,6 +110,9 @@ void sched_preempt(struct thread *td); >>> void sched_add(struct thread *td, int flags); >>> void sched_clock(struct thread *td); >>> void sched_rem(struct thread *td); >>> +#if defined(SCHED_ULE) && defined(SMP) >>> +void sched_load_rem(struct thread *td); >>> +#endif >>> void sched_tick(int cnt); >>> void sched_relinquish(struct thread *td); >>> struct thread *sched_choose(void); >>> >> >> I found another scenario in taskqueue, in the function >> taskqueue_terminate, current thread tries to wake >> another thread up and sleep immediately, the tq_mutex sometimes >> is a spinlock. So if you remove one thread load from current cpu >> before wakeup, the resumed thread may be put on same cpu, >> so it will optimize the cpu scheduling too. > > I think that in order to fit with sched_add() modifies I have in mind > (see other patches within this thread) wakeup() should grow a new > argument, or maybe we can use wakeup_flags() new KPI. > If the latter is the case, I would also propose to let wakeup_one() to > be absorbed into wakeup_flags() with its own flag. > Yes, I like the idea. > Attilio > > From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 01:51:38 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 921A41C9 for ; Wed, 7 Nov 2012 01:51:38 +0000 (UTC) (envelope-from sjg@evilcode.net) Received: from mail-ia0-f182.google.com (mail-ia0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4C2B68FC0A for ; Wed, 7 Nov 2012 01:51:37 +0000 (UTC) Received: by mail-ia0-f182.google.com with SMTP id k10so999587iag.13 for ; Tue, 06 Nov 2012 17:51:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=uV/z6mSftyXZloCTXx5kfXXsn2ttzEjpZA23ClDuiiE=; b=aDg7Y/SYiFIN+6EaiRmmH7pNSEODNIpT9wns8oVaWcc9qcfzDxXNZNmKGk3YtqBt7Z +Ukoof7wnDD8GesctkiBK3MDsJ4q4zM8psHRASJeAM0/+VCDPayp2CdDeHTCbxTgTZ7a Z4/gnhLqsHzmrPCxLkwD+hqChgmzqlxSSmFEDVIWqstktcq8FUzwxW2b/MffrQa8EU7D 7S4Ji5PNM8zgDBDG6IWsVhAc7je9v00XjdhTFbJc3K2OWF4BQdbAGFDaO7Yj77NYe4I4 donw7i7+B4R6LeFjtuSkiAr9MZHYeaf0bbLfv4YRY7tF7RZmtU29mkn7P7KPT5pdZZqT N23A== MIME-Version: 1.0 Received: by 10.50.213.34 with SMTP id np2mr3148621igc.57.1352253097368; Tue, 06 Nov 2012 17:51:37 -0800 (PST) Received: by 10.64.29.3 with HTTP; Tue, 6 Nov 2012 17:51:37 -0800 (PST) In-Reply-To: <50997AB8.40607@rawbw.com> References: <50980ADD.4010402@rawbw.com> <50994FE0.2070205@rawbw.com> <50997AB8.40607@rawbw.com> Date: Tue, 6 Nov 2012 18:51:37 -0700 Message-ID: Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? From: "Samuel J. Greear" To: Yuri Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQk0ZLZDowU1Gk5kgw4vSDOxTMDq1zSf7Q0wTEtw22/wrRQzxYlFctnUwKER1HhWlFxEGBON X-Mailman-Approved-At: Wed, 07 Nov 2012 04:32:23 +0000 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 01:51:38 -0000 On Tue, Nov 6, 2012 at 2:01 PM, Yuri wrote: > On 11/06/2012 11:10, Samuel J. Greear wrote: >> >> Single and multi-socket hardware are not really directly comparable in >> PostgreSQL tests. > > > So if the CPUs are split between sockets, would such system generally > perform better or worse with PostgeSQL vs. non-split situation? > > Yuri Unless the algorithms you are testing are able to operate entirely out of the processors caches (and PostgreSQL does not fall into that category) performance will be generally lower as you add sockets. FreeBSD's ULE scheduler is aware of this and takes it into account, but the performance ULE is able to maintain across sockets (this applies to other OS's schedulers too) is more damage control than anything else. Sam From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 06:17:48 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BBD676B2 for ; Wed, 7 Nov 2012 06:17:48 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4F3678FC16 for ; Wed, 7 Nov 2012 06:17:48 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:dc:ffe:46f:81d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 6ACE24AC1C; Wed, 7 Nov 2012 10:17:46 +0400 (MSK) Date: Wed, 7 Nov 2012 10:17:43 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1447579648.20121107101743@serebryakov.spb.ru> To: Wojciech Puchar Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: References: <50980ADD.4010402@rawbw.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Yuri , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 06:17:48 -0000 Hello, Wojciech. You wrote 6 =D0=BD=D0=BE=D1=8F=D0=B1=D1=80=D1=8F 2012 =D0=B3., 13:25:05: WP> performance in normal unix style usage - multiple different programs do= ing WP> multiple different things for multiple different users - in the same ti= me. WP> This is a case with at least 99% of users. The less than 1% that have so WP> heavy load that needs separete machine dedicated to single program doing WP> one thing In my experience, in modern world, most of computers are not true-multiuser. It is dedicated servers (DB, front-end, middle layer with something like RoR or node.js) or personal (mobile) workstations. If hardware is shared between different tasks, it is shared via hypervisor and multiple OS instances... --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 07:45:52 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A4781C2E; Wed, 7 Nov 2012 07:45:52 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 7EA558FC0C; Wed, 7 Nov 2012 07:45:52 +0000 (UTC) Received: from xyf.my.dom (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id qA77joD9020527; Wed, 7 Nov 2012 07:45:51 GMT (envelope-from davidxu@freebsd.org) Message-ID: <509A11B4.7030605@freebsd.org> Date: Wed, 07 Nov 2012 15:45:56 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:14.0) Gecko/20120822 Thunderbird/14.0 MIME-Version: 1.0 To: Jeff Roberson Subject: Re: ule+smp: small optimization for turnstile priority lending References: <50587F8D.9060102@FreeBSD.org> <505AD2A5.6060008@freebsd.org> <5099C5C9.4040703@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: attilio@freebsd.org, freebsd-hackers , Jeff Roberson , Andriy Gapon X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 07:45:52 -0000 On 2012/11/07 14:17, Jeff Roberson wrote: > On Wed, 7 Nov 2012, David Xu wrote: > >> On 2012/11/06 19:03, Attilio Rao wrote: >>> On 9/20/12, David Xu wrote: >>>> On 2012/09/18 22:05, Andriy Gapon wrote: >>>>> >>>>> Here is a snippet that demonstrates the issue on a supposedly fully >>>>> loaded >>>>> 2-processor system: >>>>> >>>>> 136794 0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid >>>>> 102818", >>>>> state:"running", attributes: prio:122 >>>>> >>>>> 136793 0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid >>>>> 111916", >>>>> state:"yielding", attributes: prio:183, wmesg:"(null)", >>>>> lockname:"(null)" >>>>> >>>>> 136792 1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 >>>>> tid >>>>> 100004", >>>>> state:"running", attributes: prio:255 >>>>> >>>>> 136791 1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load", >>>>> counter:0, >>>>> attributes: none >>>>> >>>>> 136790 1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid >>>>> 113473", >>>>> state:"blocked", attributes: prio:122, wmesg:"(null)", >>>>> lockname:"unp_mtx" >>>>> >>>>> 136789 1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load", >>>>> counter:2, >>>>> attributes: none >>>>> >>>>> 136788 1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid >>>>> 113473", >>>>> point:"wokeup", attributes: linkedto:"Xorg tid 102818" >>>>> >>>>> 136787 1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid >>>>> 102818", >>>>> state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473" >>>>> >>>>> 136786 1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load", >>>>> counter:1, >>>>> attributes: none >>>>> >>>>> 136785 1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid >>>>> 102818", >>>>> state:"runq rem", attributes: prio:176 >>>>> >>>>> 136784 1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid >>>>> 102818", >>>>> point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox >>>>> tid >>>>> 113473" >>>>> >>>>> 136783 1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid >>>>> 113473", >>>>> state:"running", attributes: prio:104 >>>>> >>>>> See how how the Xorg thread was forced from CPU 1 to CPU 0 where it >>>>> preempted >>>>> cc1plus thread (I do have preemption enabled) only to leave CPU 1 with >>>>> zero load. >>>>> >>>>> Here is a proposed solution: >>>>> >>>>> turnstile_wait: optimize priority lending to a thread on a >>>>> runqueue >>>>> >>>>> As the current thread is definitely going into mi_switch, it now >>>>> removes >>>>> its load before doing priority propagation which can potentially >>>>> result >>>>> in sched_add. In the SMP && ULE case the latter searches for >>>>> the >>>>> least loaded CPU to place a boosted thread, which is supposedly >>>>> about >>>>> to run. >>>>> >>>>> diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c >>>>> index 8e466cd..3299cae 100644 >>>>> --- a/sys/kern/sched_ule.c >>>>> +++ b/sys/kern/sched_ule.c >>>>> @@ -1878,7 +1878,10 @@ sched_switch(struct thread *td, struct thread >>>>> *newtd, int >>>>> flags) >>>>> /* This thread must be going to sleep. */ >>>>> TDQ_LOCK(tdq); >>>>> mtx = thread_lock_block(td); >>>>> - tdq_load_rem(tdq, td); >>>>> +#if defined(SMP) >>>>> + if ((flags & SW_TYPE_MASK) != SWT_TURNSTILE) >>>>> +#endif >>>>> + tdq_load_rem(tdq, td); >>>>> } >>>>> /* >>>>> * We enter here with the thread blocked and assigned to the >>>>> @@ -2412,6 +2415,21 @@ sched_rem(struct thread *td) >>>>> tdq_setlowpri(tdq, NULL); >>>>> } >>>>> >>>>> +void >>>>> +sched_load_rem(struct thread *td) >>>>> +{ >>>>> + struct tdq *tdq; >>>>> + >>>>> + KASSERT(td == curthread, >>>>> + ("sched_rem_load: only curthread is supported")); >>>>> + KASSERT(td->td_oncpu == td->td_sched->ts_cpu, >>>>> + ("thread running on cpu different from ts_cpu")); >>>>> + tdq = TDQ_CPU(td->td_sched->ts_cpu); >>>>> + TDQ_LOCK_ASSERT(tdq, MA_OWNED); >>>>> + MPASS(td->td_lock == TDQ_LOCKPTR(tdq)); >>>>> + tdq_load_rem(tdq, td); >>>>> +} >>>>> + >>>>> /* >>>>> * Fetch cpu utilization information. Updates on demand. >>>>> */ >>>>> diff --git a/sys/kern/subr_turnstile.c b/sys/kern/subr_turnstile.c >>>>> index 31d16fe..d1d68e9 100644 >>>>> --- a/sys/kern/subr_turnstile.c >>>>> +++ b/sys/kern/subr_turnstile.c >>>>> @@ -731,6 +731,13 @@ turnstile_wait(struct turnstile *ts, struct >>>>> thread >>>>> *owner, >>>>> int queue) >>>>> LIST_INSERT_HEAD(&ts->ts_free, td->td_turnstile, ts_hash); >>>>> } >>>>> thread_lock(td); >>>>> +#if defined(SCHED_ULE) && defined(SMP) >>>>> + /* >>>>> + * Remove load earlier so that it does not affect cpu selection >>>>> + * for a thread waken up due to priority lending, if any. >>>>> + */ >>>>> + sched_load_rem(td); >>>>> +#endif >>>>> thread_lock_set(td, &ts->ts_lock); >>>>> td->td_turnstile = NULL; >>>>> >>>>> diff --git a/sys/sys/sched.h b/sys/sys/sched.h >>>>> index 4b8387c..b1ead1b 100644 >>>>> --- a/sys/sys/sched.h >>>>> +++ b/sys/sys/sched.h >>>>> @@ -110,6 +110,9 @@ void sched_preempt(struct thread *td); >>>>> void sched_add(struct thread *td, int flags); >>>>> void sched_clock(struct thread *td); >>>>> void sched_rem(struct thread *td); >>>>> +#if defined(SCHED_ULE) && defined(SMP) >>>>> +void sched_load_rem(struct thread *td); >>>>> +#endif >>>>> void sched_tick(int cnt); >>>>> void sched_relinquish(struct thread *td); >>>>> struct thread *sched_choose(void); >>>>> >>>> >>>> I found another scenario in taskqueue, in the function >>>> taskqueue_terminate, current thread tries to wake >>>> another thread up and sleep immediately, the tq_mutex sometimes >>>> is a spinlock. So if you remove one thread load from current cpu >>>> before wakeup, the resumed thread may be put on same cpu, >>>> so it will optimize the cpu scheduling too. >>> >>> I think that in order to fit with sched_add() modifies I have in mind >>> (see other patches within this thread) wakeup() should grow a new >>> argument, or maybe we can use wakeup_flags() new KPI. >>> If the latter is the case, I would also propose to let wakeup_one() to >>> be absorbed into wakeup_flags() with its own flag. >>> >> >> Yes, I like the idea. > >> From ~2007 > > http://people.freebsd.org/~jeff/wakeupflags.diff > > This has some different optimizations that can be done when you have a > hinted wakeup. > wakeup_flags is a good start point. But what flags should be supported? I think WAKEUP_WILLSLEEP may be needed if the current thread will switch out as soon as possible. I see you have added WAKEUP_LOCAL and WAKEUP_TAIL. Are they for cache optimization ? both of them are arguable. WAKEUP_LOCAL increases cpu migration, not good, since one benefit of ULE is keeping thread on its old cpu, the WAKEUP_LOCAL violates the design. WAKEUP_LOCAL used in pipe code may not be correct if the pipe only has few of bytes to be transfered or receiver only eats a few bytes each time, so why bother to move other threads to local cpu ? If the other threads have many bytes cached in their old cpu, this migration is expensive. WAKEUP_TAIL is a good idea, I guess that you want to wake up hot-thread its code and data are still in its old cpu's cache. But this will lead to unfair. I'd like the sleep queue to implement a policy like I did in libthr, it picks a thread at tail of queue in a fixed percentage, it does have some level of unfair but not fatal at all. > Thanks, > Jeff From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 06:17:45 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0F6606AD for ; Wed, 7 Nov 2012 06:17:45 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id C771B8FC12 for ; Wed, 7 Nov 2012 06:17:44 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bi1so978861pad.13 for ; Tue, 06 Nov 2012 22:17:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:x-x-sender:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version:content-type:x-gm-message-state; bh=VN1hk01Xq8BVYQs6E74eurX4JSgKBFOnRMU+O7uZtsM=; b=OQMbVclRcnPsJ8ajpD5cxhj3COUftC5WRCNjizp6dRiIIipjCfiOtcsza8RUsydxCN cem/wMoe+miwkGpBOGWw1blG+ebeqYusiBBgks4XF7KV5QaQAe8zPnylkc8J2vO7gCEs 6ou1p5TSY2T/neI4XbxywjMJ16eSilSfAvxDLAhKHfwoHsBQqwszetOAqJ6kCTCz3sfn yTI4W9AKbKBMM3z8TODr4jB9RhgS9coV2+iEwcBu+CVP6iozdcf59EVehbd8EhcplCnM cXmO4ND/fACmtrMztT5Whudj14VH847tAJQUx2pYVKT8gawGCz1GbowktJMV+dXIhhV2 Vt+Q== Received: by 10.68.228.36 with SMTP id sf4mr10824677pbc.20.1352269063401; Tue, 06 Nov 2012 22:17:43 -0800 (PST) Received: from rrcs-66-91-135-210.west.biz.rr.com (rrcs-66-91-135-210.west.biz.rr.com. [66.91.135.210]) by mx.google.com with ESMTPS id ty4sm13595301pbc.57.2012.11.06.22.17.41 (version=SSLv3 cipher=OTHER); Tue, 06 Nov 2012 22:17:42 -0800 (PST) Date: Tue, 6 Nov 2012 20:17:20 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: David Xu Subject: Re: ule+smp: small optimization for turnstile priority lending In-Reply-To: <5099C5C9.4040703@freebsd.org> Message-ID: References: <50587F8D.9060102@FreeBSD.org> <505AD2A5.6060008@freebsd.org> <5099C5C9.4040703@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Gm-Message-State: ALoCoQnwzHgWWnmALnhQZHkeRcph8EX6ZXZfoANS+nZqcdKjzGs8pTTfI0dZKEWF2j0uIsThjy3M X-Mailman-Approved-At: Wed, 07 Nov 2012 12:40:11 +0000 Cc: attilio@freebsd.org, freebsd-hackers , Jeff Roberson , Andriy Gapon X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 06:17:45 -0000 On Wed, 7 Nov 2012, David Xu wrote: > On 2012/11/06 19:03, Attilio Rao wrote: >> On 9/20/12, David Xu wrote: >>> On 2012/09/18 22:05, Andriy Gapon wrote: >>>> >>>> Here is a snippet that demonstrates the issue on a supposedly fully >>>> loaded >>>> 2-processor system: >>>> >>>> 136794 0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid >>>> 102818", >>>> state:"running", attributes: prio:122 >>>> >>>> 136793 0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid >>>> 111916", >>>> state:"yielding", attributes: prio:183, wmesg:"(null)", lockname:"(null)" >>>> >>>> 136792 1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 tid >>>> 100004", >>>> state:"running", attributes: prio:255 >>>> >>>> 136791 1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load", >>>> counter:0, >>>> attributes: none >>>> >>>> 136790 1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid >>>> 113473", >>>> state:"blocked", attributes: prio:122, wmesg:"(null)", lockname:"unp_mtx" >>>> >>>> 136789 1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load", >>>> counter:2, >>>> attributes: none >>>> >>>> 136788 1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid >>>> 113473", >>>> point:"wokeup", attributes: linkedto:"Xorg tid 102818" >>>> >>>> 136787 1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid >>>> 102818", >>>> state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473" >>>> >>>> 136786 1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load", >>>> counter:1, >>>> attributes: none >>>> >>>> 136785 1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid >>>> 102818", >>>> state:"runq rem", attributes: prio:176 >>>> >>>> 136784 1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid >>>> 102818", >>>> point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox tid >>>> 113473" >>>> >>>> 136783 1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid >>>> 113473", >>>> state:"running", attributes: prio:104 >>>> >>>> See how how the Xorg thread was forced from CPU 1 to CPU 0 where it >>>> preempted >>>> cc1plus thread (I do have preemption enabled) only to leave CPU 1 with >>>> zero load. >>>> >>>> Here is a proposed solution: >>>> >>>> turnstile_wait: optimize priority lending to a thread on a runqueue >>>> >>>> As the current thread is definitely going into mi_switch, it now >>>> removes >>>> its load before doing priority propagation which can potentially >>>> result >>>> in sched_add. In the SMP && ULE case the latter searches for the >>>> least loaded CPU to place a boosted thread, which is supposedly >>>> about >>>> to run. >>>> >>>> diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c >>>> index 8e466cd..3299cae 100644 >>>> --- a/sys/kern/sched_ule.c >>>> +++ b/sys/kern/sched_ule.c >>>> @@ -1878,7 +1878,10 @@ sched_switch(struct thread *td, struct thread >>>> *newtd, int >>>> flags) >>>> /* This thread must be going to sleep. */ >>>> TDQ_LOCK(tdq); >>>> mtx = thread_lock_block(td); >>>> - tdq_load_rem(tdq, td); >>>> +#if defined(SMP) >>>> + if ((flags & SW_TYPE_MASK) != SWT_TURNSTILE) >>>> +#endif >>>> + tdq_load_rem(tdq, td); >>>> } >>>> /* >>>> * We enter here with the thread blocked and assigned to the >>>> @@ -2412,6 +2415,21 @@ sched_rem(struct thread *td) >>>> tdq_setlowpri(tdq, NULL); >>>> } >>>> >>>> +void >>>> +sched_load_rem(struct thread *td) >>>> +{ >>>> + struct tdq *tdq; >>>> + >>>> + KASSERT(td == curthread, >>>> + ("sched_rem_load: only curthread is supported")); >>>> + KASSERT(td->td_oncpu == td->td_sched->ts_cpu, >>>> + ("thread running on cpu different from ts_cpu")); >>>> + tdq = TDQ_CPU(td->td_sched->ts_cpu); >>>> + TDQ_LOCK_ASSERT(tdq, MA_OWNED); >>>> + MPASS(td->td_lock == TDQ_LOCKPTR(tdq)); >>>> + tdq_load_rem(tdq, td); >>>> +} >>>> + >>>> /* >>>> * Fetch cpu utilization information. Updates on demand. >>>> */ >>>> diff --git a/sys/kern/subr_turnstile.c b/sys/kern/subr_turnstile.c >>>> index 31d16fe..d1d68e9 100644 >>>> --- a/sys/kern/subr_turnstile.c >>>> +++ b/sys/kern/subr_turnstile.c >>>> @@ -731,6 +731,13 @@ turnstile_wait(struct turnstile *ts, struct thread >>>> *owner, >>>> int queue) >>>> LIST_INSERT_HEAD(&ts->ts_free, td->td_turnstile, ts_hash); >>>> } >>>> thread_lock(td); >>>> +#if defined(SCHED_ULE) && defined(SMP) >>>> + /* >>>> + * Remove load earlier so that it does not affect cpu selection >>>> + * for a thread waken up due to priority lending, if any. >>>> + */ >>>> + sched_load_rem(td); >>>> +#endif >>>> thread_lock_set(td, &ts->ts_lock); >>>> td->td_turnstile = NULL; >>>> >>>> diff --git a/sys/sys/sched.h b/sys/sys/sched.h >>>> index 4b8387c..b1ead1b 100644 >>>> --- a/sys/sys/sched.h >>>> +++ b/sys/sys/sched.h >>>> @@ -110,6 +110,9 @@ void sched_preempt(struct thread *td); >>>> void sched_add(struct thread *td, int flags); >>>> void sched_clock(struct thread *td); >>>> void sched_rem(struct thread *td); >>>> +#if defined(SCHED_ULE) && defined(SMP) >>>> +void sched_load_rem(struct thread *td); >>>> +#endif >>>> void sched_tick(int cnt); >>>> void sched_relinquish(struct thread *td); >>>> struct thread *sched_choose(void); >>>> >>> >>> I found another scenario in taskqueue, in the function >>> taskqueue_terminate, current thread tries to wake >>> another thread up and sleep immediately, the tq_mutex sometimes >>> is a spinlock. So if you remove one thread load from current cpu >>> before wakeup, the resumed thread may be put on same cpu, >>> so it will optimize the cpu scheduling too. >> >> I think that in order to fit with sched_add() modifies I have in mind >> (see other patches within this thread) wakeup() should grow a new >> argument, or maybe we can use wakeup_flags() new KPI. >> If the latter is the case, I would also propose to let wakeup_one() to >> be absorbed into wakeup_flags() with its own flag. >> > > Yes, I like the idea. >From ~2007 http://people.freebsd.org/~jeff/wakeupflags.diff This has some different optimizations that can be done when you have a hinted wakeup. Thanks, Jeff > >> Attilio >> >> > From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 13:47:23 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D3D2497F for ; Wed, 7 Nov 2012 13:47:23 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 3376B8FC08 for ; Wed, 7 Nov 2012 13:47:22 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA7El6Km005948; Wed, 7 Nov 2012 15:47:06 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA7EkxdV005945; Wed, 7 Nov 2012 15:46:59 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 7 Nov 2012 15:46:59 +0100 (CET) From: Wojciech Puchar To: Garrett Cooper Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: Message-ID: References: <50980ADD.4010402@rawbw.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 07 Nov 2012 15:47:14 +0100 (CET) Cc: Olivier Smedts , Yuri , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 13:47:23 -0000 >> But working on software to make it better in some kind of synthetic benchmark is common in commercial software world. ("We have more performance per buck than company X") > > "Synthetic benchmarks" as you put it shouldn't be the ultimate basis for a decision, but instead allow users to gauge whether or >not a certain software or hardware configuration is suitable for their >given workload. No more, no less. only when OS is not tuned for benchmarks. You see that given OS is great for some database test doing repetitively few operations, then you run in for YOUR workload for which OS isn't tuned and it's bad. Even if it is still database only workload. Even worse that on the same machine you do other things. From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 13:48:38 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E76D3CCF for ; Wed, 7 Nov 2012 13:48:38 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 49E748FC0A for ; Wed, 7 Nov 2012 13:48:38 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA7EmbT1005957; Wed, 7 Nov 2012 15:48:37 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA7EmbUf005954; Wed, 7 Nov 2012 15:48:37 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 7 Nov 2012 15:48:37 +0100 (CET) From: Wojciech Puchar To: Garrett Cooper Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: Message-ID: References: <50980ADD.4010402@rawbw.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 07 Nov 2012 15:48:37 +0100 (CET) Cc: Yuri , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 13:48:39 -0000 >> >> actually FreeBSD defaults are actually good for COMMON usage. and can be tuned. >> >> default MAXBSIZE is one exception. > > "Common usage" is vague. While FreeBSD might do ok for some applications (dev box, simple workstation/laptop, etc), there are other areas that require additional tuning to get better perf that arguably shouldn't as much (or there should be templates for doing so): 10GbE and mbuf and network tuning; file server and file descriptor, network tuning, etc; low latency desktop and scheduler tweaking; etc. still any idea why MAXBSIZE is 128kB by default. for modern hard disk it is a disaster. 2 or even 4 megabyte is OK. > > Not to say that freebsd is entirely at fault, but because it's more of a commodity OS that Linux, more tweaking is required... actually IMHO much more tweaking is needed with linux, at least from what i know from other people. And they are not newbies From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 13:49:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 13D3AEC2 for ; Wed, 7 Nov 2012 13:49:33 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 6BADF8FC0A for ; Wed, 7 Nov 2012 13:49:32 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA7EnTXv005966; Wed, 7 Nov 2012 15:49:29 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA7EnTLg005963; Wed, 7 Nov 2012 15:49:29 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 7 Nov 2012 15:49:29 +0100 (CET) From: Wojciech Puchar To: Yuri Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: <50997AB8.40607@rawbw.com> Message-ID: References: <50980ADD.4010402@rawbw.com> <50994FE0.2070205@rawbw.com> <50997AB8.40607@rawbw.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 07 Nov 2012 15:49:30 +0100 (CET) Cc: freebsd-hackers@freebsd.org, "Samuel J. Greear" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 13:49:33 -0000 On Tue, 6 Nov 2012, Yuri wrote: > On 11/06/2012 11:10, Samuel J. Greear wrote: >> Single and multi-socket hardware are not really directly comparable in >> PostgreSQL tests. > > So if the CPUs are split between sockets, would such system generally perform > better or worse with PostgeSQL vs. non-split situation? > multiple sockets means more interconnect delay=lower performance. From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 13:50:51 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2C3D9FCE; Wed, 7 Nov 2012 13:50:51 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 7E73D8FC17; Wed, 7 Nov 2012 13:50:50 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA7Eonr5005974; Wed, 7 Nov 2012 15:50:49 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA7EonTp005971; Wed, 7 Nov 2012 15:50:49 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 7 Nov 2012 15:50:49 +0100 (CET) From: Wojciech Puchar To: Lev Serebryakov Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: <1447579648.20121107101743@serebryakov.spb.ru> Message-ID: References: <50980ADD.4010402@rawbw.com> <1447579648.20121107101743@serebryakov.spb.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 07 Nov 2012 15:50:49 +0100 (CET) Cc: Yuri , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 13:50:51 -0000 > In my experience, in modern world, most of computers are not > true-multiuser. It is dedicated servers (DB, front-end, middle layer > with something like RoR or node.js) or personal (mobile) > workstations. If hardware is shared between different tasks, it is > shared via hypervisor and multiple OS instances... which is completely strange and inefficient. But yes i am aware that people do that. But that's not about performance at all ;) From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 14:36:04 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 44B9EBCC for ; Wed, 7 Nov 2012 14:36:04 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-ea0-f182.google.com (mail-ea0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id BFEC68FC08 for ; Wed, 7 Nov 2012 14:36:03 +0000 (UTC) Received: by mail-ea0-f182.google.com with SMTP id c10so824817eaa.13 for ; Wed, 07 Nov 2012 06:36:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=hXhJuK6yz2JwmUk+AP0bpA64By+HGz2KAIXwCZB0R84=; b=d92arrSTnWVRO3CBERELj0K6c3AuGK27QMCZew3wsk/WbB2hUhWof6esgFxOHLYcFm rpzpzZaXOO3pXtiR2LDzOLtfhnSTsIVTeO6S8yc/s9eMI/BDaTB80/PCLIL4PczYS53U QMQf+JUn0G6l/B5b7XWz/F8k6Cj21NuLRn4kAnUFQjLJtt73TLU+rSFUOs2laW9HUNym yEkZchPV/sAQa28IKwl7MNts/q+SeXDSu6bPC5rmZ8j4y5TIIhfwA4NRIqAv6d4dLg6m i25rCsv+7N8cA6pjEEA9ucFUHlbFoKEzUX8NsIjucv3efEzwf7VCMoTvZkDk4MuZRwVr GP9A== Received: by 10.14.216.193 with SMTP id g41mr16170449eep.37.1352298962614; Wed, 07 Nov 2012 06:36:02 -0800 (PST) Received: from ndenevsa.sf.moneybookers.net (g1.moneybookers.com. [217.18.249.148]) by mx.google.com with ESMTPS id g47sm64211959eeo.6.2012.11.07.06.36.01 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 07 Nov 2012 06:36:01 -0800 (PST) Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Content-Type: text/plain; charset=us-ascii From: Nikolay Denev In-Reply-To: Date: Wed, 7 Nov 2012 16:35:59 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <50980ADD.4010402@rawbw.com> To: Wojciech Puchar X-Mailer: Apple Mail (2.1499) Cc: Garrett Cooper , Yuri , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 14:36:04 -0000 On Nov 7, 2012, at 4:48 PM, Wojciech Puchar = wrote: >>>=20 >>> actually FreeBSD defaults are actually good for COMMON usage. and = can be tuned. >>>=20 >>> default MAXBSIZE is one exception. >>=20 >> "Common usage" is vague. While FreeBSD might do ok for some = applications (dev box, simple workstation/laptop, etc), there are other = areas that require additional tuning to get better perf that arguably = shouldn't as much (or there should be templates for doing so): 10GbE and = mbuf and network tuning; file server and file descriptor, network = tuning, etc; low latency desktop and scheduler tweaking; etc. >=20 > still any idea why MAXBSIZE is 128kB by default. for modern hard disk = it is a disaster. 2 or even 4 megabyte is OK. >=20 >>=20 >> Not to say that freebsd is entirely at fault, but because it's more = of a commodity OS that Linux, more tweaking is required... > actually IMHO much more tweaking is needed with linux, at least from = what i know from other people. And they are not newbies > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" Actually MAXBSIZE is 64k, MAXPHYS is 128k. There was a thread about NFS performance where it was mentioned that = bigger MAXBSIZE leads to KVA fragmentation. From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 15:11:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B622B497 for ; Wed, 7 Nov 2012 15:11:11 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id 691868FC0A for ; Wed, 7 Nov 2012 15:11:11 +0000 (UTC) Received: from [192.168.1.18] (unknown [217.157.7.221]) by csmtp2.one.com (Postfix) with ESMTPA id B976A3078444; Wed, 7 Nov 2012 15:11:03 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? From: Erik Cederstrand In-Reply-To: Date: Wed, 7 Nov 2012 16:11:03 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <47D1FF90-A96D-4620-9AA0-324865CE827B@cederstrand.dk> References: <50980ADD.4010402@rawbw.com> To: Nikolay Denev X-Mailer: Apple Mail (2.1499) Cc: Garrett Cooper , Wojciech Puchar , "freebsd-hackers@freebsd.org" , Yuri X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 15:11:11 -0000 Den 07/11/2012 kl. 15.35 skrev Nikolay Denev : > Actually MAXBSIZE is 64k, MAXPHYS is 128k. >=20 > There was a thread about NFS performance where it was mentioned that = bigger MAXBSIZE leads to KVA fragmentation. That thread starts here: = http://lists.freebsd.org/pipermail/freebsd-arch/2010-April/010143.html Erik= From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 16:01:11 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BA651601 for ; Wed, 7 Nov 2012 16:01:11 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id 980E68FC14 for ; Wed, 7 Nov 2012 16:01:11 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id qA7G1AiB020355 for hackers@freebsd.org; Wed, 7 Nov 2012 16:01:10 GMT (envelope-from kientzle@freebsd.org) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id wdyhe74eshgh35dkdwkbvvr3bi; for hackers@freebsd.org; Wed, 07 Nov 2012 16:01:10 +0000 (UTC) (envelope-from kientzle@freebsd.org) From: Tim Kientzle Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: FreeBSD on RaspberryPi Date: Wed, 7 Nov 2012 08:01:08 -0800 Message-Id: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> To: FreeBSD Hackers Mime-Version: 1.0 (Apple Message framework v1283) X-Mailer: Apple Mail (2.1283) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 16:01:11 -0000 WARNING: This is still highly experimental and by no means ready for "production use", but some folks might find it interesting. To boot FreeBSD on your RaspberryPi, you'll need: 1) A RaspberryPi. 2) A serial cable similar to this one: www.adafruit.com/products/954 3) An SD card of 2GB or larger Download this 111MB file (~1.6G uncompressed): = http://people.freebsd.org/~kientzle/FreeBSD-RPI-B-r242362-2012-10-30.img.x= z Uncompress it, dd it onto your SD card, pop it in and apply power. (The serial cable above can also provide power; just leave the red lead disconnected until you get the SD card plugged in.) KNOWN BROKEN STUFF * There's no framebuffer/syscons yet. Hence the need for a serial = cable. * The memory is mis-probed (actually a boot loader problem, not a FreeBSD kernel issue), so you'll only get to use 128MB (you might be able to change this for a single boot by breaking into ubldr and editing the FDT by hand) * There has been NO attempt to reduce the footprint of this image. It's a completely stock build of FreeBSD-CURRENT. (Actually, I have turned off sendmail and a few other things in = rc.conf, but compensated by building world with full debug enabled.) * I've personally not tried USB or Ethernet and have no idea if they = work. HOW TO BUILD YOUR OWN IMAGE The script I used to build this image is at: github.com/kientzle/freebsd-beaglebone (It was originally developed for BeagleBone.) Enjoy! Boot message (edited for length): DRAM: 128 MiB WARNING: Caches not enabled MMC: bcm2835_sdh: 0 Using default environment In: serial Out: serial Err: serial Net: Net Initialization Skipped No ethernet found. Hit any key to stop autoboot: 0=20 reading uEnv.txt 74 bytes read Importing environment from mmc ... reading ubldr 728201 bytes read ## Starting application at 0x02000054 ... Consoles: U-Boot console =20 Compatible API signature found @7b75220 Number of U-Boot devices: 1 FreeBSD/armv6 U-Boot loader, Revision 1.2 (root@fci386.localdomain, Wed Nov 7 01:54:47 PST 2012) DRAM: 128MB Device: disk - /boot/kernel/kernel data=3D0x300238+0x1ec7c = syms=3D[0x4+0x70f80+0x4+0x56d3c] Hit [Enter] to boot immediately, or any other key for command prompt. Booting [/boot/kernel/kernel]... =20 fdt_start: 0x003DC138 fdt_reg_valid(): reg#0 (start: 0x0 size: 0x8000000) valid! Kernel entry at 0x100100... Kernel args: (null) =85 Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #3: Wed Nov 7 04:23:11 PST 2012 root@fci386.localdomain:/usr/obj/arm.armv6/usr/src/sys/RPI-B arm CPU: Sheeva 88SV581x rev 7 (Marvell core) Supported features: ARM_ISA THUMB2 JAZELLE ARMv4 Security_Ext WB enabled LABT branch prediction enabled 16KB/32B 4-way instruction cache 16KB/32B 4-way write-back-locking-C data cache real memory =3D 134217728 (128 MB) avail memory =3D 125685760 (119 MB) =85=20 FreeBSD/arm (raspberry-pi) (ttyu0) login: root FreeBSD 10.0-CURRENT (RPI-B) #3: Wed Nov 7 04:23:11 PST 2012 Welcome to FreeBSD! =85 root@raspberry-pi:/root # top last pid: 490; load averages: 0.63, 0.30, 0.12 up 0+00:01:32 = 12:47:50 7 processes: 1 running, 6 sleeping CPU: 0.4% user, 0.0% nice, 0.8% system, 2.3% interrupt, 96.5% idle Mem: 59M Active, 7312K Inact, 8136K Wired, 10M Buf, 47M Free Swap:=20 PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 488 root 1 16 0 10808K 10280K pause 0:00 5.27% csh 490 root 1 41 0 10940K 10080K RUN 0:00 4.84% top 487 root 1 8 0 10980K 10164K wait 0:00 4.31% login 484 root 1 -8 0 9848K 9596K piperd 0:00 0.67% logger 452 root 1 42 0 14724K 2208K select 0:00 0.00% sshd 486 root 1 8 0 1656K 1396K nanslp 0:00 0.00% sleep 483 root 1 8 0 10416K 1128K wait 0:00 0.00% sh From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 16:09:42 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9F4AA7F4; Wed, 7 Nov 2012 16:09:42 +0000 (UTC) (envelope-from yerenkow@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id F11678FC12; Wed, 7 Nov 2012 16:09:41 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id x43so1002507wey.13 for ; Wed, 07 Nov 2012 08:09:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FaBl6wrzIjNX1uLi3Tery4YO25jsRfjo8B1aqGPA2eg=; b=sVCeTQg8PTSVXuuHPAZnLWXT1zY8+1OAdLeGJn3EklJlPRvJr2Tq1GCi3xeQI1kEzE aFLLAFkGVtoMnziu9dXDoCIxCowhRWnkryignguuT9gBbbMvRMr1PRZ/1sxlUG4fvFHH h5K5jnar3Va3kHQuFQMUXfQWaFoXBBeEgftl2k4OAp8SVq3085w7jEHZacEY/Ep87Yd9 ml8wD39feWLgilGUK+TJX4lsKvtHGzjS5P7r4H6eAQi183F8u0cqHka+k5acuxWz/gZL bFz18XD5QLxKTnjxW+EE29aqphWpCQM/6ktJ1kjKtFOs/KioRPtXEhGPyIKzu0boDYeJ D+/w== MIME-Version: 1.0 Received: by 10.216.197.142 with SMTP id t14mr2045109wen.151.1352304580475; Wed, 07 Nov 2012 08:09:40 -0800 (PST) Received: by 10.194.40.131 with HTTP; Wed, 7 Nov 2012 08:09:40 -0800 (PST) Received: by 10.194.40.131 with HTTP; Wed, 7 Nov 2012 08:09:40 -0800 (PST) In-Reply-To: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> References: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> Date: Wed, 7 Nov 2012 18:09:40 +0200 Message-ID: Subject: Re: FreeBSD on RaspberryPi From: Alexander Yerenkow To: Tim Kientzle Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 16:09:42 -0000 Such experiments was tried by me and others in August; I got framebuffer worked in rca/hdmi; usb not saw any of my mices/keyboards; ethernet worked, but produced hangs ( which render device useless).I hope this help :) Regards, Alexander Yerenkow 07.11.2012 18:01 =D0=CF=CC=D8=DA=CF=D7=C1=D4=C5=CC=D8 "Tim Kientzle" =CE=C1=D0=C9=D3=C1=CC: > WARNING: This is still highly experimental and by no > means ready for "production use", but some folks might > find it interesting. > > To boot FreeBSD on your RaspberryPi, you'll need: > 1) A RaspberryPi. > 2) A serial cable similar to this one: www.adafruit.com/products/954 > 3) An SD card of 2GB or larger > > Download this 111MB file (~1.6G uncompressed): > > http://people.freebsd.org/~kientzle/FreeBSD-RPI-B-r242362-2012-10-30.img.= xz > > Uncompress it, dd it onto your SD card, pop it in and apply power. > (The serial cable above can also provide power; just leave the red > lead disconnected until you get the SD card plugged in.) > > > KNOWN BROKEN STUFF > > * There's no framebuffer/syscons yet. Hence the need for a serial cabl= e. > > * The memory is mis-probed (actually a boot loader problem, > not a FreeBSD kernel issue), so you'll only get to use 128MB > (you might be able to change this for a single boot by breaking > into ubldr and editing the FDT by hand) > > * There has been NO attempt to reduce the footprint of this image. > It's a completely stock build of FreeBSD-CURRENT. > (Actually, I have turned off sendmail and a few other things in > rc.conf, > but compensated by building world with full debug enabled.) > > * I've personally not tried USB or Ethernet and have no idea if they > work. > > > HOW TO BUILD YOUR OWN IMAGE > > The script I used to build this image is at: > github.com/kientzle/freebsd-beaglebone > (It was originally developed for BeagleBone.) > > > Enjoy! > > > > Boot message (edited for length): > > > DRAM: 128 MiB > WARNING: Caches not enabled > MMC: bcm2835_sdh: 0 > Using default environment > > In: serial > Out: serial > Err: serial > Net: Net Initialization Skipped > No ethernet found. > Hit any key to stop autoboot: 0 > reading uEnv.txt > > 74 bytes read > Importing environment from mmc ... > reading ubldr > > 728201 bytes read > ## Starting application at 0x02000054 ... > Consoles: U-Boot console > Compatible API signature found @7b75220 > Number of U-Boot devices: 1 > > FreeBSD/armv6 U-Boot loader, Revision 1.2 > (root@fci386.localdomain, Wed Nov 7 01:54:47 PST 2012) > DRAM: 128MB > > Device: disk > - > /boot/kernel/kernel data=3D0x300238+0x1ec7c syms=3D[0x4+0x70f80+0x4+0x56d= 3c] > Hit [Enter] to boot immediately, or any other key for command prompt. > Booting [/boot/kernel/kernel]... > fdt_start: 0x003DC138 > fdt_reg_valid(): reg#0 (start: 0x0 size: 0x8000000) valid! > Kernel entry at 0x100100... > Kernel args: (null) > > ... > > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 10.0-CURRENT #3: Wed Nov 7 04:23:11 PST 2012 > root@fci386.localdomain:/usr/obj/arm.armv6/usr/src/sys/RPI-B arm > CPU: Sheeva 88SV581x rev 7 (Marvell core) > Supported features: ARM_ISA THUMB2 JAZELLE ARMv4 Security_Ext > WB enabled LABT branch prediction enabled > 16KB/32B 4-way instruction cache > 16KB/32B 4-way write-back-locking-C data cache > real memory =3D 134217728 (128 MB) > avail memory =3D 125685760 (119 MB) > > ... > > FreeBSD/arm (raspberry-pi) (ttyu0) > > login: root > FreeBSD 10.0-CURRENT (RPI-B) #3: Wed Nov 7 04:23:11 PST 2012 > > Welcome to FreeBSD! > > ... > > root@raspberry-pi:/root # top > > last pid: 490; load averages: 0.63, 0.30, 0.12 up 0+00:01:32 > 12:47:50 > 7 processes: 1 running, 6 sleeping > CPU: 0.4% user, 0.0% nice, 0.8% system, 2.3% interrupt, 96.5% idle > Mem: 59M Active, 7312K Inact, 8136K Wired, 10M Buf, 47M Free > Swap: > > PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND > 488 root 1 16 0 10808K 10280K pause 0:00 5.27% csh > 490 root 1 41 0 10940K 10080K RUN 0:00 4.84% top > 487 root 1 8 0 10980K 10164K wait 0:00 4.31% login > 484 root 1 -8 0 9848K 9596K piperd 0:00 0.67% logger > 452 root 1 42 0 14724K 2208K select 0:00 0.00% sshd > 486 root 1 8 0 1656K 1396K nanslp 0:00 0.00% sleep > 483 root 1 8 0 10416K 1128K wait 0:00 0.00% sh > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 16:42:19 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9969E988; Wed, 7 Nov 2012 16:42:19 +0000 (UTC) (envelope-from lars@e-new.0x20.net) Received: from mail.0x20.net (mail.0x20.net [IPv6:2001:aa8:fffb:1::3]) by mx1.freebsd.org (Postfix) with ESMTP id 212D18FC0C; Wed, 7 Nov 2012 16:42:18 +0000 (UTC) Received: from e-new.0x20.net (mail.0x20.net [IPv6:2001:aa8:fffb:1::3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.0x20.net (Postfix) with ESMTPS id 5675B6A6000; Wed, 7 Nov 2012 17:42:17 +0100 (CET) Received: from e-new.0x20.net (localhost [127.0.0.1]) by e-new.0x20.net (8.14.5/8.14.5) with ESMTP id qA7GgHL7001689; Wed, 7 Nov 2012 17:42:17 +0100 (CET) (envelope-from lars@e-new.0x20.net) Received: (from lars@localhost) by e-new.0x20.net (8.14.5/8.14.5/Submit) id qA7GgHrj000950; Wed, 7 Nov 2012 17:42:17 +0100 (CET) (envelope-from lars) Date: Wed, 7 Nov 2012 17:42:17 +0100 From: Lars Engels To: Tim Kientzle Subject: Re: FreeBSD on RaspberryPi Message-ID: <20121107164217.GL31744@e-new.0x20.net> References: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="eXeQRtP7FdKTPOB4" Content-Disposition: inline In-Reply-To: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> X-Editor: VIM - Vi IMproved 7.3 X-Operation-System: FreeBSD 8.3-RELEASE-p4 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 16:42:19 -0000 --eXeQRtP7FdKTPOB4 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 07, 2012 at 08:01:08AM -0800, Tim Kientzle wrote: > WARNING: This is still highly experimental and by no > means ready for "production use", but some folks might > find it interesting. >=20 > To boot FreeBSD on your RaspberryPi, you'll need: > 1) A RaspberryPi. > 2) A serial cable similar to this one: www.adafruit.com/products/954 > 3) An SD card of 2GB or larger >=20 > Download this 111MB file (~1.6G uncompressed): > http://people.freebsd.org/~kientzle/FreeBSD-RPI-B-r242362-2012-10-30.i= mg.xz >=20 > Uncompress it, dd it onto your SD card, pop it in and apply power. > (The serial cable above can also provide power; just leave the red > lead disconnected until you get the SD card plugged in.) >=20 >=20 > KNOWN BROKEN STUFF >=20 > * There's no framebuffer/syscons yet. Hence the need for a serial cabl= e. >=20 > * The memory is mis-probed (actually a boot loader problem, > not a FreeBSD kernel issue), so you'll only get to use 128MB > (you might be able to change this for a single boot by breaking > into ubldr and editing the FDT by hand) >=20 > * There has been NO attempt to reduce the footprint of this image. > It's a completely stock build of FreeBSD-CURRENT. > (Actually, I have turned off sendmail and a few other things in rc.co= nf, > but compensated by building world with full debug enabled.) >=20 > * I've personally not tried USB or Ethernet and have no idea if they wo= rk. =2E.. Are you aware of this? http://kernelnomicon.org/?p=3D178 --eXeQRtP7FdKTPOB4 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlCaj2kACgkQKc512sD3afidrACgqaO3e7ppA8MqUjoXd/+GpB4Q MoEAnjyVVi4TSsOY1eRBMmBtzcvFnPIL =JLFE -----END PGP SIGNATURE----- --eXeQRtP7FdKTPOB4-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 16:58:00 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2D96C134 for ; Wed, 7 Nov 2012 16:58:00 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 6D0648FC12 for ; Wed, 7 Nov 2012 16:57:58 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA7Hvtbf001453; Wed, 7 Nov 2012 18:57:55 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA7HvsOn001450; Wed, 7 Nov 2012 18:57:54 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 7 Nov 2012 18:57:54 +0100 (CET) From: Wojciech Puchar To: Nikolay Denev Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: Message-ID: References: <50980ADD.4010402@rawbw.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 07 Nov 2012 18:57:55 +0100 (CET) Cc: Garrett Cooper , Yuri , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 16:58:00 -0000 >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > Actually MAXBSIZE is 64k, MAXPHYS is 128k. sorry MAXPHYS > > There was a thread about NFS performance where it was mentioned that bigger MAXBSIZE leads to KVA fragmentation. NFS is never fast. but EVERYTHING is slow when operating on big files. modern SATA disk can read/write over 1MB within single seek time From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 23:24:39 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4F20CB75; Wed, 7 Nov 2012 23:24:39 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from mail.ipfw.ru (unknown [IPv6:2a01:4f8:120:6141::2]) by mx1.freebsd.org (Postfix) with ESMTP id D538F8FC0A; Wed, 7 Nov 2012 23:24:38 +0000 (UTC) Received: from v6.mpls.in ([2a02:978:2::5] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1TWF2U-000HvY-7R; Thu, 08 Nov 2012 03:28:06 +0400 Message-ID: <509AEDAC.10002@FreeBSD.org> Date: Thu, 08 Nov 2012 03:24:28 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120121 Thunderbird/9.0 MIME-Version: 1.0 To: freebsd-net@FreeBSD.org, freebsd-hackers@FreeBSD.org Subject: [patch] reducing arp locking Content-Type: multipart/mixed; boundary="------------030808050000070300050904" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 23:24:39 -0000 This is a multi-part message in MIME format. --------------030808050000070300050904 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hello list! Currently we need to acquire 2 read locks to perform simple 6-byte copying from arp record to packet ethernet header. It seems that acquiring lle lock for fast path (main traffic flow) is not necessary even with current code. My tests shows ~10% improvement with this patch applied. If nobody objects I plan to commit this change at the end of next week. --------------030808050000070300050904 Content-Type: text/plain; name="no_arp_lle_lock.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="no_arp_lle_lock.diff" Index: sys/netinet/in.c =================================================================== --- sys/netinet/in.c (revision 242524) +++ sys/netinet/in.c (working copy) @@ -1476,7 +1476,7 @@ in_lltable_lookup(struct lltable *llt, u_int flags if (LLE_IS_VALID(lle)) { if (flags & LLE_EXCLUSIVE) LLE_WLOCK(lle); - else + else if (!(flags & LLE_UNLOCKED)) LLE_RLOCK(lle); } done: Index: sys/netinet/if_ether.c =================================================================== --- sys/netinet/if_ether.c (revision 242524) +++ sys/netinet/if_ether.c (working copy) @@ -293,10 +293,10 @@ arpresolve(struct ifnet *ifp, struct rtentry *rt0, struct sockaddr *dst, u_char *desten, struct llentry **lle) { struct llentry *la = 0; - u_int flags = 0; + u_int flags = LLE_UNLOCKED; struct mbuf *curr = NULL; struct mbuf *next = NULL; - int error, renew; + int error, renew = 0; *lle = NULL; if (m != NULL) { @@ -315,7 +315,41 @@ arpresolve(struct ifnet *ifp, struct rtentry *rt0, retry: IF_AFDATA_RLOCK(ifp); la = lla_lookup(LLTABLE(ifp), flags, dst); + + /* + * Fast path. Do not require rlock on llentry. + */ + if ((la != NULL) && (flags & LLE_UNLOCKED)) { + if ((la->la_flags & LLE_VALID) && + ((la->la_flags & LLE_STATIC) || la->la_expire > time_uptime)) { + bcopy(&la->ll_addr, desten, ifp->if_addrlen); + /* + * If entry has an expiry time and it is approaching, + * see if we need to send an ARP request within this + * arpt_down interval. + */ + if (!(la->la_flags & LLE_STATIC) && + time_uptime + la->la_preempt > la->la_expire) { + renew = 1; + la->la_preempt--; + } + + IF_AFDATA_RUNLOCK(ifp); + if (renew != 0) + arprequest(ifp, NULL, &SIN(dst)->sin_addr, NULL); + + return (0); + } + + /* Revert to normal path for other cases */ + flags &= ~LLE_UNLOCKED; + *lle = la; + LLE_RNLOCK(la); + } + + IF_AFDATA_RUNLOCK(ifp); + if ((la == NULL) && ((flags & LLE_EXCLUSIVE) == 0) && ((ifp->if_flags & (IFF_NOARP | IFF_STATICARP)) == 0)) { flags |= (LLE_CREATE | LLE_EXCLUSIVE); @@ -332,25 +366,6 @@ retry: return (EINVAL); } - if ((la->la_flags & LLE_VALID) && - ((la->la_flags & LLE_STATIC) || la->la_expire > time_uptime)) { - bcopy(&la->ll_addr, desten, ifp->if_addrlen); - /* - * If entry has an expiry time and it is approaching, - * see if we need to send an ARP request within this - * arpt_down interval. - */ - if (!(la->la_flags & LLE_STATIC) && - time_uptime + la->la_preempt > la->la_expire) { - arprequest(ifp, NULL, &SIN(dst)->sin_addr, NULL); - la->la_preempt--; - } - - *lle = la; - error = 0; - goto done; - } - if (la->la_flags & LLE_STATIC) { /* should not happen! */ log(LOG_DEBUG, "arpresolve: ouch, empty static llinfo for %s\n", inet_ntoa(SIN(dst)->sin_addr)); Index: sys/net/if_llatbl.h =================================================================== --- sys/net/if_llatbl.h (revision 242524) +++ sys/net/if_llatbl.h (working copy) @@ -178,6 +178,7 @@ MALLOC_DECLARE(M_LLTABLE); #define LLE_EXCLUSIVE 0x2000 /* return lle xlocked */ #define LLE_DELETE 0x4000 /* delete on a lookup - match LLE_IFADDR */ #define LLE_CREATE 0x8000 /* create on a lookup miss */ +#define LLE_UNLOCKED 0x10000 /* return lle unlocked */ #define LLATBL_HASH(key, mask) \ (((((((key >> 8) ^ key) >> 8) ^ key) >> 8) ^ key) & mask) --------------030808050000070300050904-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 7 23:46:32 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 97D4F37F; Wed, 7 Nov 2012 23:46:32 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5A4F18FC08; Wed, 7 Nov 2012 23:46:32 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bi1so1642235pad.13 for ; Wed, 07 Nov 2012 15:46:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=1dOelbSSJQcmAOG7UkuvSb1cLBMTZyKpbOJ4iWxkMlI=; b=ja50FWPvKjt98p3THTqjRR3X6+4CV9DaqAIe1akX3P6wstOIb1j/HO7mpeRFTA51YX NdwNGrmmhBBOWyfXlqwTht9oVwnn/jgeLezzzSzplwCc/DsKXcsCQGnzUmj/UqtX6tGf tWycjPoQkK18mR3MZb8AsZAWeTtmVpOagC8p5yNRShjJMi9ysKwTFygffGUVGESAkfIW yB3EG65eWJjFqIkFZ4rP9R01vwN76Ol2uGT5aq0p3Edg/YmwI9/u7r5A5SQR7E/7Pgr0 2+MvOCy+c+goalaJlUu9pScYFxob6NeaZ1iv8dBQL79V4Dk8ntTJUlUh9FiDyGoaPvMx qarg== MIME-Version: 1.0 Received: by 10.68.137.41 with SMTP id qf9mr18231413pbb.103.1352331992011; Wed, 07 Nov 2012 15:46:32 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.68.124.130 with HTTP; Wed, 7 Nov 2012 15:46:31 -0800 (PST) In-Reply-To: <509AEDAC.10002@FreeBSD.org> References: <509AEDAC.10002@FreeBSD.org> Date: Wed, 7 Nov 2012 15:46:31 -0800 X-Google-Sender-Auth: jNar6e2WSwl_0DvDNZnoj8EG93c Message-ID: Subject: Re: [patch] reducing arp locking From: Adrian Chadd To: "Alexander V. Chernikov" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2012 23:46:32 -0000 On 7 November 2012 15:24, Alexander V. Chernikov wrote: > Hello list! > > Currently we need to acquire 2 read locks to perform simple 6-byte copying > from arp record to packet ethernet header. > > It seems that acquiring lle lock for fast path (main traffic flow) is not > necessary even with current code. > > My tests shows ~10% improvement with this patch applied. > > If nobody objects I plan to commit this change at the end of next week. That's a great catch! How'd you discover it? Adrian From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 02:54:45 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 23B31F67; Thu, 8 Nov 2012 02:54:45 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id D87258FC08; Thu, 8 Nov 2012 02:54:44 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id qA82scZe024143; Thu, 8 Nov 2012 02:54:38 GMT (envelope-from kientzle@freebsd.org) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id 23apg4d63ka64g3mffqt4u767e; Thu, 08 Nov 2012 02:54:38 +0000 (UTC) (envelope-from kientzle@freebsd.org) Subject: Re: FreeBSD on RaspberryPi Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Tim Kientzle In-Reply-To: <20121107164217.GL31744@e-new.0x20.net> Date: Wed, 7 Nov 2012 18:54:38 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <35720413-A16F-4C44-AAB4-DE3DE4D11744@freebsd.org> References: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> <20121107164217.GL31744@e-new.0x20.net> To: Lars Engels X-Mailer: Apple Mail (2.1283) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 02:54:45 -0000 On Nov 7, 2012, at 8:42 AM, Lars Engels wrote: > On Wed, Nov 07, 2012 at 08:01:08AM -0800, Tim Kientzle wrote: >> WARNING: This is still highly experimental and by no >> means ready for "production use", but some folks might >> find it interesting. >>=20 >> To boot FreeBSD on your RaspberryPi, you'll need: >> 1) A RaspberryPi. >> 2) A serial cable similar to this one: = www.adafruit.com/products/954 >> 3) An SD card of 2GB or larger >>=20 >> Download this 111MB file (~1.6G uncompressed): >> = http://people.freebsd.org/~kientzle/FreeBSD-RPI-B-r242362-2012-10-30.img.x= z >>=20 > ... >=20 > Are you aware of this? >=20 > http://kernelnomicon.org/?p=3D178 Yes, of course. The image I posted wouldn't have been possible without their excellent work. Tim From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 05:16:03 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4731542C for ; Thu, 8 Nov 2012 05:16:03 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id 03CAB8FC08 for ; Thu, 8 Nov 2012 05:16:02 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id qA85G0Cb025011; Thu, 8 Nov 2012 05:16:00 GMT (envelope-from kientzle@freebsd.org) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id bdxr3as9rvbqmsu78aaav79ymw; Thu, 08 Nov 2012 05:16:00 +0000 (UTC) (envelope-from kientzle@freebsd.org) Subject: Re: FreeBSD on RaspberryPi Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=windows-1251 From: Tim Kientzle In-Reply-To: Date: Wed, 7 Nov 2012 21:16:00 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <872D3322-C4BB-4636-B4FC-6C065CE4A45F@freebsd.org> References: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> To: Alexander Yerenkow X-Mailer: Apple Mail (2.1283) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 05:16:03 -0000 On Nov 7, 2012, at 8:09 AM, Alexander Yerenkow wrote: > Such experiments was tried by me and others in August; I got = framebuffer worked in rca/hdmi =85. How? I haven't seen the drivers for that yet. Tim From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 07:06:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 42B7B2B9 for ; Thu, 8 Nov 2012 07:06:11 +0000 (UTC) (envelope-from sse.auburn.study@gmail.com) Received: from mail-ia0-f182.google.com (mail-ia0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0860D8FC15 for ; Thu, 8 Nov 2012 07:06:10 +0000 (UTC) Received: by mail-ia0-f182.google.com with SMTP id k10so2242891iag.13 for ; Wed, 07 Nov 2012 23:06:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Y+bivWLN7aufpwpW8iaL1NOboEdmrTUIXggZeP7gKFg=; b=zPZCuH/LwjaV9T4GiQx6RqHr6pvavXhXN7yZX46iHS+YcBoClLzcv47Ut55zTWMc1M WlY9dKedgmY8CpAwVDAeDOYiXKIzaF4dJ9v/B+rAlgfkoJtCgxEa7IwvxfSPPpRqya0z Zdi/mua549WYlfN8Eu8aFlTyg3wFhnOzODZr5ekCXDH/ih6phCoqQWZbF0cYRRQ2kAOP /hINlUcx0BQ0OB5USJne9E8Ut+IcTAPYSwO8/ZV4OB3EvAJ4rblGt/97bqKpSPdNx31Q lQ2Qd5YJ9ilqYcM2R0L14PH3DpJOCcNq/Ejug3mCzeWqVqpSN8GgYhnLwFK3WXTXITD0 mzyQ== MIME-Version: 1.0 Received: by 10.50.140.106 with SMTP id rf10mr18976644igb.48.1352358370215; Wed, 07 Nov 2012 23:06:10 -0800 (PST) Received: by 10.64.6.168 with HTTP; Wed, 7 Nov 2012 23:06:10 -0800 (PST) Date: Thu, 8 Nov 2012 01:06:10 -0600 Message-ID: Subject: Buffer Overflow Study at Auburn University - FreeBSD developers I would really appreciate your help! From: Auburn Study To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 07:06:11 -0000 Hi All, I am a graduate student at Auburn University, working with Dr. Munawar Hafiz. We are working on an empirical study project to understand the software engineering practices used in companies that produce secure software. In particular, we are concentrating on how developers write code to prevent buffer overflow and integer overflow vulnerabilities. We are interested in the software development process: how you develop software, how you test and analyze programs to detect vulnerabilities, and what processes you follow to remove bugs. We are looking into automated tools that software developers use, and are expecting that there is a common insight in the security engineering process that can be reusable. We request your assistance by participating in this research study. We would greatly appreciate it if you would share your experience with us by answering the questions at the end of this email. We may send some follow up questions based on your response in future. Your response(s) will be kept confidential, and will only be aggregated with those of other responders. Please let us know if you have any questions or concerns regarding the study. Thanks in advance for your support. Yasmeen Rawajfih Software Analysis, Transformations and Security Group Auburn University Working under the supervision of: Dr. Munawar Hafiz Assistant Professor Dept. of Computer Science and Software Engineering Auburn University Auburn, AL http://munawarhafiz.com/ Questions: (There are ten questions.) 1. How long have you been a software developer? 2. How long have you been affiliated with FreeBSD? Were you part of the original development team for this software? 3. What is the size of the current code base? 4. Did you follow a coding standard when developing this software? Is it a standard determined by your group? 5. What did you use to manage bug reports in your software? Does it satisfy your requirements? Are there other software options that you would consider switching to? 6. Did you use any compiler options to detect integer overflow vulnerabilities? Do you think that they are useful? 7. Did you use any automated (static or dynamic analysis) tools to detect buffer overflows, integer overflows, or any other bugs? Which tools did you use? Why these tools? 8. Did you use fuzzing? Which tools did you use and why? If you wrote your own fuzzer, why did you write it yourself? Was it written from scratch or by extending some other fuzzing tools? 9. Did you have specific phases during development where you concentrated on fixing security issues? Did you have a test suite, unit tests, or regression tests? 10. Buffer overflows often result from the use of unsafe functions, such as strcpy. Does your software use those? If you use a different string library, why is it used? Is it an in-house library or an off-the-shelf library? Did you migrate your code to use the string library? From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 07:48:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4299EA89 for ; Thu, 8 Nov 2012 07:48:33 +0000 (UTC) (envelope-from s.dave.jones@gmail.com) Received: from mail-wg0-f50.google.com (mail-wg0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id C52258FC0C for ; Thu, 8 Nov 2012 07:48:32 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id 16so1639798wgi.31 for ; Wed, 07 Nov 2012 23:48:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=uJI5mTRza52EN53tarLIq6U28fZPiGoqEP9RDXuhX3I=; b=fpbTWr62h+H9MMM6CRyD11H9rcbydrHuEe1y0NjY3rg79NQg54asI8v7zhFJWvdSVg JwcwdO4n6OZmV9t6yUwwUdHvGYwOZv7oLe6WIBJQ+qAIOsH8PcJoiJrlc1oCxWL8Gn8Q Pssdj+KRYxlIJ+uuCIioy4hm54lspMpKyc3H12iRF72y46Bn4cm4DuoTpgmskaIFDlNM z053Vq81obmlxkmHXcEAXIJbsk3kk7mf5Gl91V8o9SZH1tjO+LCRql+ZXMhYBXfZKUkg rnAG9KzVRpRq5yAK6Q60Bha1IJKcyQ+WLogLxaOF2hM/aaXu3K4byx8jKljZWhvINSTR jQYQ== MIME-Version: 1.0 Received: by 10.180.84.202 with SMTP id b10mr11609975wiz.13.1352360911488; Wed, 07 Nov 2012 23:48:31 -0800 (PST) Received: by 10.194.56.233 with HTTP; Wed, 7 Nov 2012 23:48:31 -0800 (PST) Date: Thu, 8 Nov 2012 15:48:31 +0800 Message-ID: Subject: A question about creating a system call From: dave jones To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 07:48:33 -0000 Hello, I know how to create system calls, but I'm a bit confused about sys/kern/syscalls.master file explained. For example, if I have a foo system call, following code is added: 532 AUE_NULL STD { int foo(char *str); } The question is in column two AUE_NULL, can I replace it with AUE_FOO? How to determine the system call should be audit or not? Thank you. Regards, Dave. From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 09:38:24 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 30F52B07 for ; Thu, 8 Nov 2012 09:38:24 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id D63778FC08 for ; Thu, 8 Nov 2012 09:38:23 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:b53e:a763:8923:37c0]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 69F214AC1C; Thu, 8 Nov 2012 13:38:15 +0400 (MSK) Date: Thu, 8 Nov 2012 13:38:11 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1961607438.20121108133811@serebryakov.spb.ru> To: Erik Cederstrand Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: <47D1FF90-A96D-4620-9AA0-324865CE827B@cederstrand.dk> References: <50980ADD.4010402@rawbw.com> <47D1FF90-A96D-4620-9AA0-324865CE827B@cederstrand.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Garrett Cooper , Nikolay Denev , Yuri , "freebsd-hackers@freebsd.org" , Wojciech Puchar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 09:38:24 -0000 Hello, Erik. You wrote 7 =D0=BD=D0=BE=D1=8F=D0=B1=D1=80=D1=8F 2012 =D0=B3., 19:11:03: EC> That thread starts here: EC> http://lists.freebsd.org/pipermail/freebsd-arch/2010-April/010143.html Year 2010! And we still limited by MAXPHYS (128K) transfers :( --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 10:27:57 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BE3CA27A; Thu, 8 Nov 2012 10:27:57 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from mail.ipfw.ru (unknown [IPv6:2a01:4f8:120:6141::2]) by mx1.freebsd.org (Postfix) with ESMTP id 544CB8FC0C; Thu, 8 Nov 2012 10:27:57 +0000 (UTC) Received: from dhcp170-36-red.yandex.net ([95.108.170.36]) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1TWPOO-000OgF-Ip; Thu, 08 Nov 2012 14:31:24 +0400 Message-ID: <509B88B1.3070905@FreeBSD.org> Date: Thu, 08 Nov 2012 14:25:53 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120627 Thunderbird/13.0.1 MIME-Version: 1.0 To: Andre Oppermann Subject: Re: [patch] reducing arp locking References: <509AEDAC.10002@FreeBSD.org> <509B884F.7040106@networx.ch> In-Reply-To: <509B884F.7040106@networx.ch> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@FreeBSD.org, freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 10:27:57 -0000 On 08.11.2012 14:24, Andre Oppermann wrote: > On 08.11.2012 00:24, Alexander V. Chernikov wrote: >> Hello list! >> >> Currently we need to acquire 2 read locks to perform simple 6-byte >> copying from arp record to packet >> ethernet header. >> >> It seems that acquiring lle lock for fast path (main traffic flow) is >> not necessary even with >> current code. >> >> My tests shows ~10% improvement with this patch applied. >> >> If nobody objects I plan to commit this change at the end of next week. > > This is risky and prone to race conditions. The copy of the MAC address > should be done while the table read lock is held to protect against the It is done exactly as you say: table read lock is held. > entry going away. You can either return with table lock held and drop > it after the copy, or you could a modified lookup function that takes a > pointer for the copy destination, do the copy with the read lock, and then > return. If no entry is found an error is returned and obviously no copy > is done. > -- WBR, Alexander From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 10:48:28 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 30140CD6 for ; Thu, 8 Nov 2012 10:48:28 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 83E3C8FC12 for ; Thu, 8 Nov 2012 10:48:27 +0000 (UTC) Received: (qmail 64365 invoked from network); 8 Nov 2012 12:23:34 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 8 Nov 2012 12:23:34 -0000 Message-ID: <509B8DF5.5010101@freebsd.org> Date: Thu, 08 Nov 2012 11:48:21 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: "Alexander V. Chernikov" Subject: Re: [patch] reducing arp locking References: <509AEDAC.10002@FreeBSD.org> <509B884F.7040106@networx.ch> <509B88B1.3070905@FreeBSD.org> In-Reply-To: <509B88B1.3070905@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@FreeBSD.org, freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 10:48:28 -0000 On 08.11.2012 11:25, Alexander V. Chernikov wrote: > On 08.11.2012 14:24, Andre Oppermann wrote: >> On 08.11.2012 00:24, Alexander V. Chernikov wrote: >>> Hello list! >>> >>> Currently we need to acquire 2 read locks to perform simple 6-byte >>> copying from arp record to packet >>> ethernet header. >>> >>> It seems that acquiring lle lock for fast path (main traffic flow) is >>> not necessary even with >>> current code. >>> >>> My tests shows ~10% improvement with this patch applied. >>> >>> If nobody objects I plan to commit this change at the end of next week. >> >> This is risky and prone to race conditions. The copy of the MAC address >> should be done while the table read lock is held to protect against the > It is done exactly as you say: table read lock is held. Right. Sorry. I didn't immediately get that the IF_AFDATA_LOCK is the table lock. -- Andre >> entry going away. You can either return with table lock held and drop >> it after the copy, or you could a modified lookup function that takes a >> pointer for the copy destination, do the copy with the read lock, and then >> return. If no entry is found an error is returned and obviously no copy >> is done. >> > > From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 10:56:44 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C4E012A6; Thu, 8 Nov 2012 10:56:44 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 191008FC08; Thu, 8 Nov 2012 10:56:43 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qA8AuajO009445; Thu, 8 Nov 2012 11:56:36 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qA8AuZVl009442; Thu, 8 Nov 2012 11:56:35 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Thu, 8 Nov 2012 11:56:35 +0100 (CET) From: Wojciech Puchar To: Lev Serebryakov Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? In-Reply-To: <1961607438.20121108133811@serebryakov.spb.ru> Message-ID: References: <50980ADD.4010402@rawbw.com> <47D1FF90-A96D-4620-9AA0-324865CE827B@cederstrand.dk> <1961607438.20121108133811@serebryakov.spb.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Thu, 08 Nov 2012 11:56:36 +0100 (CET) Cc: Garrett Cooper , Nikolay Denev , Yuri , Erik Cederstrand , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 10:56:44 -0000 > EC> That thread starts here: > EC> http://lists.freebsd.org/pipermail/freebsd-arch/2010-April/010143.html > Year 2010! And we still limited by MAXPHYS (128K) transfers :( put options MAXPHYS=2097152 in your kernel config. EVERYTHING works in all production machines for over a year the only exception is my laptop with OCZ petrol SSD that hangs on any transfer >1MB, i've set it to 0.5MB here. From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 10:24:22 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F253F10D for ; Thu, 8 Nov 2012 10:24:22 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 598238FC16 for ; Thu, 8 Nov 2012 10:24:21 +0000 (UTC) Received: (qmail 64230 invoked from network); 8 Nov 2012 11:59:29 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 8 Nov 2012 11:59:29 -0000 Message-ID: <509B884F.7040106@networx.ch> Date: Thu, 08 Nov 2012 11:24:15 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: "Alexander V. Chernikov" Subject: Re: [patch] reducing arp locking References: <509AEDAC.10002@FreeBSD.org> In-Reply-To: <509AEDAC.10002@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 08 Nov 2012 12:34:31 +0000 Cc: freebsd-net@FreeBSD.org, freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 10:24:23 -0000 On 08.11.2012 00:24, Alexander V. Chernikov wrote: > Hello list! > > Currently we need to acquire 2 read locks to perform simple 6-byte copying from arp record to packet > ethernet header. > > It seems that acquiring lle lock for fast path (main traffic flow) is not necessary even with > current code. > > My tests shows ~10% improvement with this patch applied. > > If nobody objects I plan to commit this change at the end of next week. This is risky and prone to race conditions. The copy of the MAC address should be done while the table read lock is held to protect against the entry going away. You can either return with table lock held and drop it after the copy, or you could a modified lookup function that takes a pointer for the copy destination, do the copy with the read lock, and then return. If no entry is found an error is returned and obviously no copy is done. -- Andre From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 15:34:41 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 08860903; Thu, 8 Nov 2012 15:34:41 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-ea0-f182.google.com (mail-ea0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4D6E58FC0A; Thu, 8 Nov 2012 15:34:40 +0000 (UTC) Received: by mail-ea0-f182.google.com with SMTP id c10so1433843eaa.13 for ; Thu, 08 Nov 2012 07:34:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=TvTiJgaonrt5OI1MQt5Js4JyaAs3ZiU8Y4FlXulq5ok=; b=TzZq14vVJjl8HCIF+Uz+uHaJ+nS5bIeFN0Lj80d9jtbSRmgBa0tu+0eZ9mScUtDeaO taBudIvbRmVeNNkfb98D0AYJMtoeJhavXPYh4/gKJgfmXfBy/hbrLNTmxAMtEafjtpDf KakRBOWWOrcZVSra5XpiianeciR+qqUP8H7y2quf14fs++Y0zPJ4DSZLOjL8WX2WaFiI zvwjr2FHIIwZHkVVLooBtaipjT+una9SYCTRCUph5Asm+dge96sJwyMPV+LEjvVk3bMd Yjhozqs1LHuytO4AJJzAI4iw7FN5Un2kn/usN7KkuCeRerCZ+oG97fmPJi7Qiq5hFj0a QyOw== Received: by 10.14.219.2 with SMTP id l2mr28895574eep.3.1352388879158; Thu, 08 Nov 2012 07:34:39 -0800 (PST) Received: from [10.0.0.86] ([93.152.184.10]) by mx.google.com with ESMTPS id i1sm70908690eeo.8.2012.11.08.07.34.37 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 08 Nov 2012 07:34:38 -0800 (PST) Subject: Re: pgbench performance is lagging compared to Linux and DragonflyBSD? Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Content-Type: text/plain; charset=us-ascii From: Nikolay Denev In-Reply-To: Date: Thu, 8 Nov 2012 17:34:35 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <11BFE0CE-E773-4585-AB8E-2325A8DCD8B2@gmail.com> References: <50980ADD.4010402@rawbw.com> <47D1FF90-A96D-4620-9AA0-324865CE827B@cederstrand.dk> <1961607438.20121108133811@serebryakov.spb.ru> To: Wojciech Puchar X-Mailer: Apple Mail (2.1499) Cc: Garrett Cooper , "freebsd-hackers@freebsd.org" , Lev Serebryakov , Erik Cederstrand , Yuri X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 15:34:41 -0000 On Nov 8, 2012, at 12:56 PM, Wojciech Puchar = wrote: >> EC> That thread starts here: >> EC> = http://lists.freebsd.org/pipermail/freebsd-arch/2010-April/010143.html >> Year 2010! And we still limited by MAXPHYS (128K) transfers :( > put > options MAXPHYS=3D2097152 > in your kernel config. >=20 > EVERYTHING works in all production machines for over a year >=20 >=20 > the only exception is my laptop with OCZ petrol SSD that hangs on any = transfer >1MB, i've set it to 0.5MB here. Have you measured the performance increase? I'm also interested in bigger MAXBSIZE as this is what the NFS server = uses as maximum transfer size. Linux and Solaris can do up to 1MB. From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 17:13:26 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E6852102; Thu, 8 Nov 2012 17:13:26 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 91ACC8FC12; Thu, 8 Nov 2012 17:13:26 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id wc20so3867316obb.13 for ; Thu, 08 Nov 2012 09:13:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vQ60bJBJeIQqsPJiWkI/pBUgcryyrzcwumpqQL7c5mE=; b=xydsGT1bVKaC4nxouiuyz44lioUOddrrANVTgqyG3cwyR4oRMeG60UuCUVE4KVkOYx tLr03U+C6kxBqW74seAHC/1lFv8XBaf2HonqNvBfbeu887Fe7sB1x8/nU011BlDrBF9j wqy8xjjcokDSCsv95ZzppYcSnl24FOUZeBTifLScXncJKsZyC/4xgSMFEZIzl/YPLWnZ glSsWJ6korCrw+RHDTw6Fq9Yy/Sar0za3gksjXteCg116WkCGv83J58JAmvwEYlbo3yd P40ALq6+dGBcPAMHh7vK66JPLgDZakSVNINmgte1nu5I3Whm8HI7glhTrmmOWdSPxWm7 CJmA== MIME-Version: 1.0 Received: by 10.182.64.14 with SMTP id k14mr6358968obs.72.1352394805866; Thu, 08 Nov 2012 09:13:25 -0800 (PST) Received: by 10.182.133.40 with HTTP; Thu, 8 Nov 2012 09:13:25 -0800 (PST) In-Reply-To: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> References: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> Date: Thu, 8 Nov 2012 19:13:25 +0200 Message-ID: Subject: Re: FreeBSD on RaspberryPi From: Sami Halabi To: Tim Kientzle Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 17:13:27 -0000 Hi, why the console cable is needed ? Sami On Wed, Nov 7, 2012 at 6:01 PM, Tim Kientzle wrote: > WARNING: This is still highly experimental and by no > means ready for "production use", but some folks might > find it interesting. > > To boot FreeBSD on your RaspberryPi, you'll need: > 1) A RaspberryPi. > 2) A serial cable similar to this one: www.adafruit.com/products/954 > 3) An SD card of 2GB or larger > > Download this 111MB file (~1.6G uncompressed): > > http://people.freebsd.org/~kientzle/FreeBSD-RPI-B-r242362-2012-10-30.img.= xz > > Uncompress it, dd it onto your SD card, pop it in and apply power. > (The serial cable above can also provide power; just leave the red > lead disconnected until you get the SD card plugged in.) > > > KNOWN BROKEN STUFF > > * There's no framebuffer/syscons yet. Hence the need for a serial cabl= e. > > * The memory is mis-probed (actually a boot loader problem, > not a FreeBSD kernel issue), so you'll only get to use 128MB > (you might be able to change this for a single boot by breaking > into ubldr and editing the FDT by hand) > > * There has been NO attempt to reduce the footprint of this image. > It's a completely stock build of FreeBSD-CURRENT. > (Actually, I have turned off sendmail and a few other things in > rc.conf, > but compensated by building world with full debug enabled.) > > * I've personally not tried USB or Ethernet and have no idea if they > work. > > > HOW TO BUILD YOUR OWN IMAGE > > The script I used to build this image is at: > github.com/kientzle/freebsd-beaglebone > (It was originally developed for BeagleBone.) > > > Enjoy! > > > > Boot message (edited for length): > > > DRAM: 128 MiB > WARNING: Caches not enabled > MMC: bcm2835_sdh: 0 > Using default environment > > In: serial > Out: serial > Err: serial > Net: Net Initialization Skipped > No ethernet found. > Hit any key to stop autoboot: 0 > reading uEnv.txt > > 74 bytes read > Importing environment from mmc ... > reading ubldr > > 728201 bytes read > ## Starting application at 0x02000054 ... > Consoles: U-Boot console > Compatible API signature found @7b75220 > Number of U-Boot devices: 1 > > FreeBSD/armv6 U-Boot loader, Revision 1.2 > (root@fci386.localdomain, Wed Nov 7 01:54:47 PST 2012) > DRAM: 128MB > > Device: disk > - > /boot/kernel/kernel data=3D0x300238+0x1ec7c syms=3D[0x4+0x70f80+0x4+0x56d= 3c] > Hit [Enter] to boot immediately, or any other key for command prompt. > Booting [/boot/kernel/kernel]... > fdt_start: 0x003DC138 > fdt_reg_valid(): reg#0 (start: 0x0 size: 0x8000000) valid! > Kernel entry at 0x100100... > Kernel args: (null) > > =85 > > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 10.0-CURRENT #3: Wed Nov 7 04:23:11 PST 2012 > root@fci386.localdomain:/usr/obj/arm.armv6/usr/src/sys/RPI-B arm > CPU: Sheeva 88SV581x rev 7 (Marvell core) > Supported features: ARM_ISA THUMB2 JAZELLE ARMv4 Security_Ext > WB enabled LABT branch prediction enabled > 16KB/32B 4-way instruction cache > 16KB/32B 4-way write-back-locking-C data cache > real memory =3D 134217728 (128 MB) > avail memory =3D 125685760 (119 MB) > > =85 > > FreeBSD/arm (raspberry-pi) (ttyu0) > > login: root > FreeBSD 10.0-CURRENT (RPI-B) #3: Wed Nov 7 04:23:11 PST 2012 > > Welcome to FreeBSD! > > =85 > > root@raspberry-pi:/root # top > > last pid: 490; load averages: 0.63, 0.30, 0.12 up 0+00:01:32 > 12:47:50 > 7 processes: 1 running, 6 sleeping > CPU: 0.4% user, 0.0% nice, 0.8% system, 2.3% interrupt, 96.5% idle > Mem: 59M Active, 7312K Inact, 8136K Wired, 10M Buf, 47M Free > Swap: > > PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND > 488 root 1 16 0 10808K 10280K pause 0:00 5.27% csh > 490 root 1 41 0 10940K 10080K RUN 0:00 4.84% top > 487 root 1 8 0 10980K 10164K wait 0:00 4.31% login > 484 root 1 -8 0 9848K 9596K piperd 0:00 0.67% logger > 452 root 1 42 0 14724K 2208K select 0:00 0.00% sshd > 486 root 1 8 0 1656K 1396K nanslp 0:00 0.00% sleep > 483 root 1 8 0 10416K 1128K wait 0:00 0.00% sh > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > --=20 Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 20:08:13 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4F041A31 for ; Thu, 8 Nov 2012 20:08:13 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 22C098FC08 for ; Thu, 8 Nov 2012 20:08:13 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 2854146B2E; Thu, 8 Nov 2012 15:08:12 -0500 (EST) Date: Thu, 8 Nov 2012 20:08:12 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: dave jones Subject: Re: A question about creating a system call In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 20:08:13 -0000 Hi Dave: This wiki page may be of value: http://wiki.freebsd.org/AddingAuditEvents Robert N M Watson Computer Laboratory University of Cambridge On Thu, 8 Nov 2012, dave jones wrote: > Hello, > > I know how to create system calls, but I'm a bit confused about > sys/kern/syscalls.master file explained. For example, if I have a > foo system call, following code is added: > > 532 AUE_NULL STD { int foo(char *str); } > > The question is in column two AUE_NULL, can I replace it with AUE_FOO? > How to determine the system call should be audit or not? Thank you. > > Regards, > Dave. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 20:43:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C137269C for ; Thu, 8 Nov 2012 20:43:33 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 38CB58FC0C for ; Thu, 8 Nov 2012 20:43:33 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id b5so3206761lbd.13 for ; Thu, 08 Nov 2012 12:43:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=iISXg8A31/USBXXae/XQt24yPGdpfZ61fQ13Vn7F2QQ=; b=Ooi8NG2jdQtDEvXrukLx76ToqkTk0i2A/CqjpuL/Zfi0nUTMlXeGhp9QlLTdPxHlsi AjH9M6muI77W1zG2usQQajbgLImeOyqxPBMiBszJW8bcWYGS/LajpB+dUxEGytP+EMYx KbiRO932dfd9u/c12EGzbkDbIkqcc3OCQa/RE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:x-gm-message-state; bh=iISXg8A31/USBXXae/XQt24yPGdpfZ61fQ13Vn7F2QQ=; b=Dbro4qWrBApNYOyL0M2OYzQGHkxiuiEm7I6dUEDqIqHapd5AzIQ3VM8ZRYfxIg015N Hka9D84xmEHOf6ikhwBxp/tsTGnxhUs49k/P43f0yMyYtoZTZqIbVkAgnfEz+MJZ9TvV LNf9oKEJR6BRHDSmSWlkhqp4310A6st/suwmgbnEegjKYeKwp2tjLzt2KHRa69DC+NZo fpFrzXUsMTxX56FO+bMoQ/8IcSNnaNZGTd0ls6EnFYzKmJ3LoAR4L+g9EgHS4KW4xL1P MyNdwDeN/7Ys0MAc7qrZ0+uaEMxUu+QcW4wRWSGGTG7jVpuHdJwYm6kC0oh+OXT12kaE kn/A== Received: by 10.152.131.200 with SMTP id oo8mr8641823lab.34.1352407412198; Thu, 08 Nov 2012 12:43:32 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.25.166 with HTTP; Thu, 8 Nov 2012 12:43:01 -0800 (PST) In-Reply-To: References: From: Eitan Adler Date: Thu, 8 Nov 2012 15:43:01 -0500 Message-ID: Subject: Re: -lpthread vs -pthread: does -D_REENTRANT matter? To: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQnrOQQyrfwlFpNPyOMOHaPvmSljc97R0F8yFwTpQv8O7X7tZPDvJrarDy5vOghUrlHrgVLj X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 20:43:33 -0000 On 8 October 2012 12:17, Eitan Adler wrote: > The only difference between -lpthread and -pthread that I could see is > that the latter also sets -D_REENTRANT. > However, I can't find any uses of _REENTRANT anywhere outside of a few > utilities that seem to define it manually. > > Testing with various manually written pthread programs resulted in > identical binaries, let alone identical results. > > Is there an actual difference between -pthread and -lpthread or is > this just a historical artifact? does anyone know the answer to this question? I've been experimenting but can't find a difference at all. -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 21:48:22 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D476126F; Thu, 8 Nov 2012 21:48:22 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from mail.ipfw.ru (unknown [IPv6:2a01:4f8:120:6141::2]) by mx1.freebsd.org (Postfix) with ESMTP id 8D7458FC08; Thu, 8 Nov 2012 21:48:22 +0000 (UTC) Received: from v6.mpls.in ([2a02:978:2::5] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1TWa0r-0003QI-UJ; Fri, 09 Nov 2012 01:51:50 +0400 Message-ID: <509C2899.8010701@FreeBSD.org> Date: Fri, 09 Nov 2012 01:48:09 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120121 Thunderbird/9.0 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: [patch] reducing arp locking References: <509AEDAC.10002@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 21:48:23 -0000 On 08.11.2012 03:46, Adrian Chadd wrote: > On 7 November 2012 15:24, Alexander V. Chernikov wrote: >> Hello list! >> >> Currently we need to acquire 2 read locks to perform simple 6-byte copying >> from arp record to packet ethernet header. >> >> It seems that acquiring lle lock for fast path (main traffic flow) is not >> necessary even with current code. >> >> My tests shows ~10% improvement with this patch applied. >> >> If nobody objects I plan to commit this change at the end of next week. > > That's a great catch! How'd you discover it? We have lots of FreeBSD routers doing 10G firewalling, so we're very much concerned with forwarding/firewalling performance, constantly looking for something to optimize :) > > > > Adrian > From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 22:07:17 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CDA898B6 for ; Thu, 8 Nov 2012 22:07:17 +0000 (UTC) (envelope-from sfourman@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5A5F68FC0C for ; Thu, 8 Nov 2012 22:07:17 +0000 (UTC) Received: by mail-ee0-f54.google.com with SMTP id c50so2371597eek.13 for ; Thu, 08 Nov 2012 14:07:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=JoXWUU1deBcFEWL0s8Nc0b6aLwaeOSILkB/OeKKJ4Jw=; b=HsZX5e1MNh8k5eqaGpfSbQO9tdqjMz7uoAfuYLp13KkPqp2NdeaJGUZ/BnKH/lhCoV MpT8dYcY5IoRs7uAFsNqWVP9XkUUzOHoO5kMAYUbI7EvB+y+f1Gxsa3C1UtUnvAdb823 HmF1QmiLhJD+szz3CbCvzGT7pP8IvRSxocqzFZ9jcwlSZzc45W5ix5vAx8LVnQUNVlvK rfym1/H1QBuAmgJiRNz+pncGHu3m5NwJLqESZ12cu3uPM9FvirJuGBdp7gZIs6ybZ2RL Y/oVt2pv7lYkgpAdwdQ5WmcySJnvvBoTxCW8GtO9sCmhPfZey8ybEGkStKBTiO0l9su2 Zq6Q== MIME-Version: 1.0 Received: by 10.14.214.2 with SMTP id b2mr31482484eep.32.1352412436116; Thu, 08 Nov 2012 14:07:16 -0800 (PST) Received: by 10.14.140.79 with HTTP; Thu, 8 Nov 2012 14:07:16 -0800 (PST) Date: Thu, 8 Nov 2012 16:07:16 -0600 Message-ID: Subject: Custom FreeBSD usb memstick From: "Sam Fourman Jr." To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 22:07:17 -0000 hello hackers@ I have a interest in playing around with the scripts that create the memstick image when you run make release... can anyone point me in the right direction, how would I go about modifying the size of the partition that gets created on the memstick image -- Sam Fourman Jr. From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 22:11:45 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F1336B66 for ; Thu, 8 Nov 2012 22:11:45 +0000 (UTC) (envelope-from Dave.Robison@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) by mx1.freebsd.org (Postfix) with ESMTP id B28708FC0A for ; Thu, 8 Nov 2012 22:11:45 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.31]) by ltcfislmsgpa07.fnfis.com (8.14.5/8.14.5) with ESMTP id qA8MBd0T018002 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) for ; Thu, 8 Nov 2012 16:11:39 -0600 Received: from lefty.vicor.com (10.14.152.62) by smtp.fisglobal.com (10.132.206.31) with Microsoft SMTP Server (TLS) id 14.2.309.2; Thu, 8 Nov 2012 16:11:38 -0600 Message-ID: <509C2E17.9000406@fisglobal.com> Date: Thu, 8 Nov 2012 14:11:35 -0800 From: "Robison, Dave" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:16.0) Gecko/20121013 Thunderbird/16.0.1 MIME-Version: 1.0 To: Subject: Re: Custom FreeBSD usb memstick References: In-Reply-To: X-Enigmail-Version: 1.4.5 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-Originating-IP: [10.14.152.62] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8185, 1.0.431, 0.0.0000 definitions=2012-11-08_04:2012-11-08,2012-11-08,1970-01-01 signatures=0 X-Mailman-Approved-At: Thu, 08 Nov 2012 22:33:56 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: david.robison@fisglobal.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 22:11:46 -0000 If you're just looking for a release on memstick, check out druidbsd http://druidbsd.sourceforge.net/ If you're doing it to learn the process, I'm sure you can learn a few things from druidbsd anyway... Enjoy Dave On 11/08/2012 14:07, Sam Fourman Jr. wrote: > hello hackers@ > > I have a interest in playing around with the scripts that create the > memstick image when you run make release... > can anyone point me in the right direction, how would I go about > modifying the size of the partition that gets created on the memstick > image > -- Dave Robison Sales Solution Architect II FIS Banking Solutions 510/621-2089 (w) 530/518-5194 (c) 510/621-2020 (f) daver@vicor.com david.robison@fisglobal.com _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 23:34:35 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 50596F92; Thu, 8 Nov 2012 23:34:35 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id 126F58FC0A; Thu, 8 Nov 2012 23:34:34 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bi1so2513668pad.13 for ; Thu, 08 Nov 2012 15:34:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=URQTbeENgXzGY55hA2nMfhJn87xUmnN4dLd+ppBblfM=; b=gBTkcdxkaw2tPkwHKHskDXIHljpGeABoh680jNEd6fmQDNlVHAuTEbUTkt7RIw1U8r Md7Bdue8qgIM7NbwQdBPC/PP01SAV3ymX7Y2s9k4PRdleGsO1cAc2TZ9oTRzYBtIPTSC NppMQ5rYV6RS7UOVgK3resW3tff+P8wysl+zCBkPSrOgS9RD3CBUgFeBllPdjpPCC335 SfOy6pxu5mkm28XTAC+kmjWwxSiANZbJyi81FwYqw+ucbCoZM6V2ygbZOxF0iqjuPKCF Vg5DjEprXCrW4IXn9gq7pVqeQM7h98bv8r4O/z/El78s/Prm4qh6T2EiXjhyCaePfawl 0Nsg== MIME-Version: 1.0 Received: by 10.68.238.199 with SMTP id vm7mr27994955pbc.105.1352417674573; Thu, 08 Nov 2012 15:34:34 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.68.124.130 with HTTP; Thu, 8 Nov 2012 15:34:34 -0800 (PST) In-Reply-To: <509C2899.8010701@FreeBSD.org> References: <509AEDAC.10002@FreeBSD.org> <509C2899.8010701@FreeBSD.org> Date: Thu, 8 Nov 2012 15:34:34 -0800 X-Google-Sender-Auth: d6ANbmtUjZmxzKr_bHId9NUGox8 Message-ID: Subject: Re: [patch] reducing arp locking From: Adrian Chadd To: "Alexander V. Chernikov" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 23:34:35 -0000 On 8 November 2012 13:48, Alexander V. Chernikov wrote: >> That's a great catch! How'd you discover it? > > We have lots of FreeBSD routers doing 10G firewalling, so we're very much > concerned with forwarding/firewalling performance, constantly looking for > something to optimize :) I mean, how'd you go about finding it? Adrian From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 00:06:19 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D944312E for ; Fri, 9 Nov 2012 00:06:19 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) by mx1.freebsd.org (Postfix) with ESMTP id 928248FC0A for ; Fri, 9 Nov 2012 00:06:19 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.17]) by ltcfislmsgpa03.fnfis.com (8.14.5/8.14.5) with ESMTP id qA906Ito020062 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Thu, 8 Nov 2012 18:06:18 -0600 Received: from [10.0.0.101] (10.14.152.61) by smtp.fisglobal.com (10.132.206.17) with Microsoft SMTP Server (TLS) id 14.2.309.2; Thu, 8 Nov 2012 18:06:17 -0600 Subject: Re: Custom FreeBSD usb memstick MIME-Version: 1.0 (Apple Message framework v1283) From: Devin Teske In-Reply-To: <509C4213.3090106@fisglobal.com> Date: Thu, 8 Nov 2012 16:06:16 -0800 Message-ID: <4DE05C40-9AAB-4680-AB5F-D67C29BE7778@fisglobal.com> References: <509C2E17.9000406@fisglobal.com> <509C4213.3090106@fisglobal.com> To: "Sam Fourman Jr." X-Mailer: Apple Mail (2.1283) X-Originating-IP: [10.14.152.61] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8185, 1.0.431, 0.0.0000 definitions=2012-11-08_10:2012-11-08,2012-11-08,1970-01-01 signatures=0 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers , Dave Robison X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 00:06:20 -0000 Slowly working my way to HEAD. 9.0-R is the latest I've done (and still wai= ting patiently for 9.1-R). Druid is unique in that you can create a custom USB memstick that also can = be burned to CD or DVD (with zero modifications). When getting started, keep in mind that you can do these steps on Mac OS X,= Linux, Cygwin, and (uh) FreeBSD :) 1. Find/Browse the code here=85 http://druidbsd.cvs.sourceforge.net/viewvc/druidbsd/druidbsd/druidbsd/ 2. Get the code=85 In Tarball form (22.6MB): http://druidbsd.cvs.sourceforge.net/viewvc/druidbsd/druidbsd/druidbsd/?view= =3Dtar or Instructions on how to download with CVS: http://sourceforge.net/projects/druidbsd/develop NOTE: When logging in anonymously, simply press ENTER when prompted for a p= assword NOTE: Use a modulename of druidbsd/druidbsd when checking out the code NOTE: Downloaded module measures about 30MB in size 3. Execute the following: cd druidbsd ./configure NOTE: If you went the CVS route, the first command is instead: cd druidbsd/= druidbsd 4. Dependency checks: In the output of "configure" from step 3 above (which is rather small), mak= e sure no commands have been mapped "/bin/false" (except for "false" itself= , naturally). Basically, you'll need GNU make, mkisofs, and a few basic shell utilities. 5. [Optional] Customize: If you want to customize the kernel that is used, it's in "mdroot/kernels/" NOTE: If you change the name of the kernel, edit the file "mdroot/boot/menu= .rc" (kernel paths are at the top) NOTE: You can load multiple kernels if you want (and configure the menu to = display them for selection) If you want to customize the boot menu, that's in "mdroot/boot/menu.rc" If you want to add kernel modules, those go into "mdroot/boot/modules/" If you want to customize the mfsroot, there's a whole framework for that in= "dep/freebsd/mfsroot/" (however, you'll need a FreeBSD host to customize t= hat portion). If you want to simply add new commands to DruidBSD, dump new binaries into = "src/freebsd/rescue/" NOTE: There's a corresponding "src/freebsd/rescue/lib/" for you to dump lib= rary dependencies. If you want to add other miscellaneous files, just dump them into "src/free= bsd/". 6. Produce an ISO for mastering to USB stick: make NOTE: Sorry, this has to be GNU make (so on FreeBSD, say "gmake" instead). NOTE: This produces the ISO file (image) of your custom FreeBSD memstick 7. Play Use the instructions here to get your ISO onto physical media: http://druidbsd.cvs.sourceforge.net/viewvc/druidbsd/druidbsd/druid/src/tool= s/ NOTE: Start with "README" NOTE: There are many methods for many operating systems documented and even= some tools for download (such as a Win32 GUI tool for imaging the ISO onto= USB stick). Remember, that ISO can be burned to optical media _and_ imaged to USB stick= (or any hard disk for that matter) without changing anything about the ISO= (that's the nature of the DRUID architecture). I demonstrated much of this at the last DevSummit, but it's not part of Fre= eBSD. If you have any questions, let me know. --=20 Devin On Nov 8, 2012, at 3:36 PM, Robison, Dave wrote: > Druid is Devin's baby so I will CC him >=20 > I do not think he has done HEAD on druid yet... but it's not a bad idea. >=20 >=20 > Dave >=20 >=20 > On 11/08/2012 15:35, Sam Fourman Jr. wrote: >> On Thu, Nov 8, 2012 at 4:11 PM, Robison, Dave >> wrote: >>> If you're just looking for a release on memstick, check out druidbsd >>>=20 >>> http://druidbsd.sourceforge.net/ >>>=20 >>> If you're doing it to learn the process, I'm sure you can learn a few >>> things from druidbsd anyway... >>>=20 >>> Enjoy >>>=20 >>> Dave >>=20 >> Thank you Dave, >>=20 >> do you build a version of DruidBSD based on HEAD? >>=20 >> im really trying to learn the process, and id like to know how to >> alter the install image size.. then ill start with customization from >> there. >>=20 >=20 >=20 > --=20 > Dave Robison > Sales Solution Architect II > FIS Banking Solutions > 510/621-2089 (w) > 530/518-5194 (c) > 510/621-2020 (f) > daver@vicor.com > david.robison@fisglobal.com _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you. From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 00:22:57 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CDE7D59F for ; Fri, 9 Nov 2012 00:22:57 +0000 (UTC) (envelope-from if@xip.at) Received: from chile.gbit.at (ns1.xip.at [193.239.188.99]) by mx1.freebsd.org (Postfix) with ESMTP id 58A178FC13 for ; Fri, 9 Nov 2012 00:22:56 +0000 (UTC) Received: (qmail 3379 invoked from network); 9 Nov 2012 01:22:55 +0100 Received: from fw.xip.at (HELO filebunker.xip.at) (89.207.145.147) by chile.gbit.at with (DHE-RSA-AES256-SHA encrypted) SMTP; 9 Nov 2012 01:22:55 +0100 Date: Fri, 9 Nov 2012 01:22:54 +0100 (CET) From: Ingo Flaschberger To: "Alexander V. Chernikov" Subject: Re: [patch] reducing arp locking In-Reply-To: <509AEDAC.10002@FreeBSD.org> Message-ID: References: <509AEDAC.10002@FreeBSD.org> User-Agent: Alpine 2.00 (LRH 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Mailman-Approved-At: Fri, 09 Nov 2012 00:30:16 +0000 Cc: freebsd-net@FreeBSD.org, freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 00:22:57 -0000 Dear Alexander, > If nobody objects I plan to commit this change at the end of next week. LLE_RNLOCK(la); should be LLE_RLOCK(la); in arpresolve Kind regards, Ingo Flaschberger From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 00:34:11 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2EFA6A42 for ; Fri, 9 Nov 2012 00:34:11 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 98AE78FC08 for ; Fri, 9 Nov 2012 00:34:10 +0000 (UTC) Received: from ur.dons.net.au (ppp118-210-218-164.lns20.adl6.internode.on.net [118.210.218.164]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id qA90Xtuk060298 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 9 Nov 2012 11:04:01 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Custom FreeBSD usb memstick From: "Daniel O'Connor" In-Reply-To: Date: Fri, 9 Nov 2012 11:03:55 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: "Sam Fourman Jr." X-Mailer: Apple Mail (2.1499) X-Spam-Score: 0.163 () BAYES_00,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 00:34:11 -0000 On 09/11/2012, at 8:37, Sam Fourman Jr. wrote: > I have a interest in playing around with the scripts that create the > memstick image when you run make release... > can anyone point me in the right direction, how would I go about > modifying the size of the partition that gets created on the memstick > image It uses makefs to create an image that is just the right size for the = files that are included. I wrote a script that will format a given device and splat the installer = stuff on it, ie cat >make-usb.sh < = ${TMPDIR}/mnt/etc/fstab sync umount "${TMPDIR}/mnt" rm -rf "${TMPDIR}" EOF The usage example is specific to my work - I have a big tarball full of = preinstalled ports which the script copies to the USB key along with a = script to install it, but you don't need that, just run.. sh ./make-usb.sh /dev/da1 /usr/obj/usr/src/release/release (Obviously /dev/da1 should be your USB key, check dmesg etc etc) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 04:45:03 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 75165B0D for ; Fri, 9 Nov 2012 04:45:03 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id 4B4AD8FC0C for ; Fri, 9 Nov 2012 04:45:03 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id qA94iusc032984; Fri, 9 Nov 2012 04:44:56 GMT (envelope-from kientzle@freebsd.org) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id hpbdpxdwk423cag3njvvuqwb9i; Fri, 09 Nov 2012 04:44:56 +0000 (UTC) (envelope-from kientzle@freebsd.org) Subject: Re: FreeBSD on RaspberryPi Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=windows-1252 From: Tim Kientzle In-Reply-To: Date: Thu, 8 Nov 2012 20:44:55 -0800 Content-Transfer-Encoding: 7bit Message-Id: References: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> To: Sami Halabi X-Mailer: Apple Mail (2.1283) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 04:45:03 -0000 > On Wed, Nov 7, 2012 at 6:01 PM, Tim Kientzle wrote: > WARNING: This is still highly experimental and by no > means ready for "production use", ... > > To boot FreeBSD on your RaspberryPi, you'll need: > 1) A RaspberryPi. > 2) A serial cable similar to this one: www.adafruit.com/products/954 On Nov 8, 2012, at 9:13 AM, Sami Halabi wrote: > > why the console cable is needed ? > As far as I can tell, the code in FreeBSD-CURRENT does not yet support the video out. So you need a serial console cable to interact with it. You might be able to interact via SSH but that requires a little bit more setup (a root password needs to be set and you need to edit /etc/ssh/sshd_config to allow root logins). If someone knows how to get the video out to work, I'm very interested! Tim From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 07:12:40 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 74BAA85A; Fri, 9 Nov 2012 07:12:40 +0000 (UTC) (envelope-from yerenkow@gmail.com) Received: from mail-wi0-f170.google.com (mail-wi0-f170.google.com [209.85.212.170]) by mx1.freebsd.org (Postfix) with ESMTP id C9C578FC12; Fri, 9 Nov 2012 07:12:39 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id hm9so368021wib.1 for ; Thu, 08 Nov 2012 23:12:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=KD/+VG/s5nPB5hh12MviAWvULG6o24vIJguBIcok/rk=; b=vbKU4WyQRK3wRQHIFwBgdme9ETyEBdwqlWoxp+G0rReylqi+r0oQHZGEcRimvb+CHe zlRpjHdFDvD3NXdgucy7wX+xHE9FaULRcz3+MdL/1jUzljVISdohADT4rWucu1MJEaEl OUjKRsp6xHsH3Ho8CwmIBmFQMcao7kPzteBXx50aVJvsghdDPoX65TtaVaPTz20BbMvB 2KmVRnD6Frk2v0Ne0wrEOHlIq28hbJVp/H8IQ3trfUhLrxKql2KFdPCti3BhTe5IaDQv coByp6NR06p0WiD9zcArWdrDmxf6DuCbeD3uLOcJyiZLeVGkXvYdb7hLKcRoOZXFc9tY KjSw== MIME-Version: 1.0 Received: by 10.181.11.197 with SMTP id ek5mr1023476wid.4.1352445153248; Thu, 08 Nov 2012 23:12:33 -0800 (PST) Received: by 10.194.40.131 with HTTP; Thu, 8 Nov 2012 23:12:33 -0800 (PST) Received: by 10.194.40.131 with HTTP; Thu, 8 Nov 2012 23:12:33 -0800 (PST) In-Reply-To: <4A5E03E5-3295-4FD4-9A06-7D1C4E9E0C12@freebsd.org> References: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> <4A5E03E5-3295-4FD4-9A06-7D1C4E9E0C12@freebsd.org> Date: Fri, 9 Nov 2012 09:12:33 +0200 Message-ID: Subject: Re: FreeBSD on RaspberryPi From: Alexander Yerenkow To: Tim Kientzle , FreeBSD Hackers Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 07:12:40 -0000 It was plain current with plain RPIB kernel config, and for graphic you should uncomment there partition about sysconsole; serial then disabled; Also, if you want ethernet - it's ue device, which also worked, but produced hangs for me in past (Hans IIRC already fixed.this). I'll have some time this weekend, feel free to contact me by gtalk or else, I will play around with my rpi with both serials and vide modes. Regards, Alexander Yerenkow 09.11.2012 6:58 =D0=CF=CC=D8=DA=CF=D7=C1=D4=C5=CC=D8 "Tim Kientzle" =CE=C1=D0=C9=D3=C1=CC: > > On Nov 7, 2012, at 8:09 AM, Alexander Yerenkow wrote: > > > Such experiments was tried by me and others in August; I got framebuffe= r > worked in rca/hdmi; ... > > Where is that code? > > Has anyone merged it to FreeBSD-CURRENT yet? > > Tim > > From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 08:42:40 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8A584D3C for ; Fri, 9 Nov 2012 08:42:40 +0000 (UTC) (envelope-from se@freebsd.org) Received: from nm18-vm0.bullet.mail.ukl.yahoo.com (nm18-vm0.bullet.mail.ukl.yahoo.com [217.146.183.95]) by mx1.freebsd.org (Postfix) with ESMTP id F133D8FC0A for ; Fri, 9 Nov 2012 08:42:39 +0000 (UTC) Received: from [217.146.183.211] by nm18.bullet.mail.ukl.yahoo.com with NNFMP; 09 Nov 2012 08:42:33 -0000 Received: from [217.146.183.108] by tm4.bullet.mail.ukl.yahoo.com with NNFMP; 09 Nov 2012 08:42:33 -0000 Received: from [127.0.0.1] by smtp109.mail.ukl.yahoo.com with NNFMP; 09 Nov 2012 08:42:33 -0000 X-Yahoo-Newman-Id: 155262.14143.bm@smtp109.mail.ukl.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 3nItc.AVM1lgLUpc1scjjpS6oIdsHJ8ukzSjZ0SrzXs5RBk aE2UI22EqTvpMkfKjrQO6u._.4U.YnS5SYktSTWRzWtKPOVhFDIXMJxzy.PB B3YlzYumAkncgnWnrINHVDG832D0ozr6Zi6MuDcYzr2K_ZM8inQbYA1SClEe N4rdh_g7dEmo_CBh3qERYnGjgiUTPvdsrWzqY_a.bLciawquoLnR81FeNRnU 2clYLHJ_z4X3dkRa6d.zE7p5LwAO_0J46nWsadlZFIyH1rNcMZxWAgA3sZTX 8x2U5QbSkDcLyAj3NrkTEZRLjwcqY7Bdp51geIazcPM7v6JODWIxWLWLyR2K tReQYe_0YMTQ.ZOa8ubU.NlrWUBEXAmwmdzWkUcal_kM9j0NeseDVADZzUxp ze6qoS3pEsJKVfQgWO1U5BtHtUP2YZfgcrYgRQbIKeaAQKMLQ3zOz0HJ5oW0 8txAFGitNBZGEfA.1.peZ8I6p5SkaFdQUHrEp.2PqgswV2aF7OLkpxPX4egu 043zQsvLG1mgbMhB_YNmZMPq3G8rOduukb3DVujWYkNoNxyWw2.dR0Yrdnz6 ESecMWN53X9TaQ36GhS1khzEhN.N8xHXrpfN5FOH.6.Ti X-Yahoo-SMTP: iDf2N9.swBDAhYEh7VHfpgq0lnq. Received: from [192.168.119.11] (se@87.158.17.230 with plain) by smtp109.mail.ukl.yahoo.com with SMTP; 09 Nov 2012 00:42:33 -0800 PST Message-ID: <509CC1F6.1010308@freebsd.org> Date: Fri, 09 Nov 2012 09:42:30 +0100 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121026 Thunderbird/16.0.2 MIME-Version: 1.0 To: Tim Kientzle , freebsd-hackers@freebsd.org Subject: Re: FreeBSD on RaspberryPi References: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.4.5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 08:42:40 -0000 Am 09.11.2012 05:44, schrieb Tim Kientzle: >> On Wed, Nov 7, 2012 at 6:01 PM, Tim Kientzle wrote: >> WARNING: This is still highly experimental and by no >> means ready for "production use", ... >> >> To boot FreeBSD on your RaspberryPi, you'll need: >> 1) A RaspberryPi. >> 2) A serial cable similar to this one: www.adafruit.com/products/954 > > > On Nov 8, 2012, at 9:13 AM, Sami Halabi wrote: >> >> why the console cable is needed ? >> > As far as I can tell, the code in FreeBSD-CURRENT > does not yet support the video out. So you need > a serial console cable to interact with it. All it takes to get the framebuffer working is that the hash chars are removed. I.e. the following works: device sc device kbdmux options SC_DFLT_FONT # compile font in makeoptions SC_DFLT_FONT=cp437 > You might be able to interact via SSH but > that requires a little bit more setup (a root > password needs to be set and you need to > edit /etc/ssh/sshd_config to allow root logins). I used SSH, and the framebuffer helps to see how far the boot process has come. It takes about 60 seconds to generate the SSH host keys, for example. [The following points are not specific to the R-PI, and I'm sure you know them, but I list them for others that may want to use their R-PI without serial console.] In order to use SSH I modified sshd_config to accept direct root logins. The root password must be set (best method: "vipw -d /mnt/etc", else you must remember to invoke "pwd_mkdb -d /mnt/etc" when you are done). The host name and IP address should be set in rc.conf (or assigned via DHCP). If you do not want to enable direct root login, then a non-privileged account in group wheel is required to be able to "su" to root. That's all I remember ... I used the build script from "http://kernelnomicon.org/?p=164" with one slight modification (tar -x ... --no-same-owner ...). My R-PI kernel contains MSDOSFS and NFS client support to allow it to mount its boot partition and NFS exported /usr/src, /usr/obj, /usr/ports and /usr/work (where I build ports). Most of them are R/O mounts. I have not tried to build world on the R-PI (cross building is so much faster ...). But ports can be build, if a swap partition is available (e.g. on SD card or via NFS - I did not try to mount a USB stick, but that might be another option). Regards, STefan From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 08:51:50 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6665D184; Fri, 9 Nov 2012 08:51:50 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id 268918FC0A; Fri, 9 Nov 2012 08:51:49 +0000 (UTC) Received: from [10.2.1.1] (unknown [10.2.1.1]) by work.netasq.com (Postfix) with ESMTPSA id 3F4A12705266; Fri, 9 Nov 2012 09:51:48 +0100 (CET) Subject: Re: [patch] reducing arp locking Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-1 From: Fabien Thomas In-Reply-To: <509B88B1.3070905@FreeBSD.org> Date: Fri, 9 Nov 2012 09:51:47 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <49EE4F42-6162-40F4-9DE0-1ACA1289B225@netasq.com> References: <509AEDAC.10002@FreeBSD.org> <509B884F.7040106@networx.ch> <509B88B1.3070905@FreeBSD.org> To: Alexander V. Chernikov X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@FreeBSD.org, Andre Oppermann , freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 08:51:50 -0000 Le 8 nov. 2012 =E0 11:25, Alexander V. Chernikov a =E9crit : > On 08.11.2012 14:24, Andre Oppermann wrote: >> On 08.11.2012 00:24, Alexander V. Chernikov wrote: >>> Hello list! >>>=20 >>> Currently we need to acquire 2 read locks to perform simple 6-byte >>> copying from arp record to packet >>> ethernet header. >>>=20 >>> It seems that acquiring lle lock for fast path (main traffic flow) = is >>> not necessary even with >>> current code. >>>=20 >>> My tests shows ~10% improvement with this patch applied. >>>=20 >>> If nobody objects I plan to commit this change at the end of next = week. >>=20 >> This is risky and prone to race conditions. The copy of the MAC = address >> should be done while the table read lock is held to protect against = the > It is done exactly as you say: table read lock is held. How do you protect from entry update if i've a ref to the entry ? You can end up doing bcopy of a partial mac address. la_preempt modification is also write access to an unlocked structure. >=20 >> entry going away. You can either return with table lock held and = drop >> it after the copy, or you could a modified lookup function that takes = a >> pointer for the copy destination, do the copy with the read lock, and = then >> return. If no entry is found an error is returned and obviously no = copy >> is done. >>=20 >=20 >=20 > --=20 > WBR, Alexander >=20 >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 09:06:13 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8798973F; Fri, 9 Nov 2012 09:06:13 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from mail.ipfw.ru (unknown [IPv6:2a01:4f8:120:6141::2]) by mx1.freebsd.org (Postfix) with ESMTP id 198848FC0C; Fri, 9 Nov 2012 09:06:13 +0000 (UTC) Received: from v6.mpls.in ([2a02:978:2::5] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1TWkap-000Akh-VY; Fri, 09 Nov 2012 13:09:40 +0400 Message-ID: <509CC776.9010200@FreeBSD.org> Date: Fri, 09 Nov 2012 13:05:58 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120121 Thunderbird/9.0 MIME-Version: 1.0 To: Fabien Thomas Subject: Re: [patch] reducing arp locking References: <509AEDAC.10002@FreeBSD.org> <509B884F.7040106@networx.ch> <509B88B1.3070905@FreeBSD.org> <49EE4F42-6162-40F4-9DE0-1ACA1289B225@netasq.com> In-Reply-To: <49EE4F42-6162-40F4-9DE0-1ACA1289B225@netasq.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net@FreeBSD.org, Andre Oppermann , freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 09:06:13 -0000 On 09.11.2012 12:51, Fabien Thomas wrote: > > Le 8 nov. 2012 à 11:25, Alexander V. Chernikov a écrit : > >> On 08.11.2012 14:24, Andre Oppermann wrote: >>> On 08.11.2012 00:24, Alexander V. Chernikov wrote: >>>> Hello list! >>>> >>>> Currently we need to acquire 2 read locks to perform simple 6-byte >>>> copying from arp record to packet >>>> ethernet header. >>>> >>>> It seems that acquiring lle lock for fast path (main traffic flow) is >>>> not necessary even with >>>> current code. >>>> >>>> My tests shows ~10% improvement with this patch applied. >>>> >>>> If nobody objects I plan to commit this change at the end of next week. >>> >>> This is risky and prone to race conditions. The copy of the MAC address >>> should be done while the table read lock is held to protect against the >> It is done exactly as you say: table read lock is held. > > How do you protect from entry update if i've a ref to the entry ? > You can end up doing bcopy of a partial mac address. I see no problems in copying incorrect mac address in that case: if host mac address id updated, this is, most likely, another host, and several packets being lost changes nothing. However, there can be some realistic scenario where this can be the case (L2 load balancing/failover). I'll update in_arpinput() to do lle removal/insertion in that case. > la_preempt modification is also write access to an unlocked structure. This one changes nothing: current code does this under _read_ lock. > > >> >>> entry going away. You can either return with table lock held and drop >>> it after the copy, or you could a modified lookup function that takes a >>> pointer for the copy destination, do the copy with the read lock, and then >>> return. If no entry is found an error is returned and obviously no copy >>> is done. >>> >> >> >> -- >> WBR, Alexander >> >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 09:52:18 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AB0722E3 for ; Fri, 9 Nov 2012 09:52:18 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id DD2B88FC14 for ; Fri, 9 Nov 2012 09:52:16 +0000 (UTC) Received: (qmail 72107 invoked from network); 9 Nov 2012 11:27:12 -0000 Received: from unknown (HELO [62.48.0.94]) ([62.48.0.94]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 9 Nov 2012 11:27:12 -0000 Message-ID: <509CD2A8.3070404@freebsd.org> Date: Fri, 09 Nov 2012 10:53:44 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121026 Thunderbird/16.0.2 MIME-Version: 1.0 To: David Xu Subject: Re: ule+smp: small optimization for turnstile priority lending References: <50587F8D.9060102@FreeBSD.org> <505AD2A5.6060008@freebsd.org> <5099C5C9.4040703@freebsd.org> <509A11B4.7030605@freebsd.org> In-Reply-To: <509A11B4.7030605@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: attilio@freebsd.org, freebsd-hackers , Jeff Roberson , Jeff Roberson , Andriy Gapon X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 09:52:18 -0000 On 07.11.2012 08:45, David Xu wrote: > On 2012/11/07 14:17, Jeff Roberson wrote: >> On Wed, 7 Nov 2012, David Xu wrote: >> >>> On 2012/11/06 19:03, Attilio Rao wrote: >>>> On 9/20/12, David Xu wrote: >>>>> I found another scenario in taskqueue, in the function >>>>> taskqueue_terminate, current thread tries to wake >>>>> another thread up and sleep immediately, the tq_mutex sometimes >>>>> is a spinlock. So if you remove one thread load from current cpu >>>>> before wakeup, the resumed thread may be put on same cpu, >>>>> so it will optimize the cpu scheduling too. >>>> >>>> I think that in order to fit with sched_add() modifies I have in mind >>>> (see other patches within this thread) wakeup() should grow a new >>>> argument, or maybe we can use wakeup_flags() new KPI. >>>> If the latter is the case, I would also propose to let wakeup_one() to >>>> be absorbed into wakeup_flags() with its own flag. >>>> >>> >>> Yes, I like the idea. >> >>> From ~2007 >> >> http://people.freebsd.org/~jeff/wakeupflags.diff >> >> This has some different optimizations that can be done when you have a >> hinted wakeup. >> > > wakeup_flags is a good start point. > But what flags should be supported? I think WAKEUP_WILLSLEEP may be > needed if the current thread will switch out as soon as possible. WAKEUP_ONE is important for accept sockets. We don't want all idle threads to compete and contend on a new connection. -- Andre > I see you have added WAKEUP_LOCAL and WAKEUP_TAIL. Are they for cache > optimization ? both of them are arguable. > > WAKEUP_LOCAL increases cpu migration, not good, since one benefit of > ULE is keeping thread on its old cpu, the WAKEUP_LOCAL violates the > design. > WAKEUP_LOCAL used in pipe code may not be correct if the pipe only > has few of bytes to be transfered or receiver only eats a few bytes > each time, so why bother to move other threads to local cpu ? > If the other threads have many bytes cached in their old cpu, this > migration is expensive. > > WAKEUP_TAIL is a good idea, I guess that you want to wake up hot-thread > its code and data are still in its old cpu's cache. But this will lead > to unfair. I'd like the sleep queue to implement a policy like I did > in libthr, it picks a thread at tail of queue in a fixed percentage, it > does have some level of unfair but not fatal at all. > >> Thanks, >> Jeff > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 09:59:22 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0225649C; Fri, 9 Nov 2012 09:59:21 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id B15918FC12; Fri, 9 Nov 2012 09:59:21 +0000 (UTC) Received: from [10.2.1.1] (unknown [10.2.1.1]) by work.netasq.com (Postfix) with ESMTPSA id D30ED2705341; Fri, 9 Nov 2012 10:59:19 +0100 (CET) Subject: Re: [patch] reducing arp locking Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-1 From: Fabien Thomas In-Reply-To: <509CC776.9010200@FreeBSD.org> Date: Fri, 9 Nov 2012 10:59:18 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <37E1F76F-D951-4B36-AF00-039DA1CC5CF3@netasq.com> References: <509AEDAC.10002@FreeBSD.org> <509B884F.7040106@networx.ch> <509B88B1.3070905@FreeBSD.org> <49EE4F42-6162-40F4-9DE0-1ACA1289B225@netasq.com> <509CC776.9010200@FreeBSD.org> To: "Alexander V. Chernikov" X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@FreeBSD.org, Andre Oppermann , freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 09:59:22 -0000 Le 9 nov. 2012 =E0 10:05, Alexander V. Chernikov a =E9crit : > On 09.11.2012 12:51, Fabien Thomas wrote: >>=20 >> Le 8 nov. 2012 =E0 11:25, Alexander V. Chernikov a =E9crit : >>=20 >>> On 08.11.2012 14:24, Andre Oppermann wrote: >>>> On 08.11.2012 00:24, Alexander V. Chernikov wrote: >>>>> Hello list! >>>>>=20 >>>>> Currently we need to acquire 2 read locks to perform simple 6-byte >>>>> copying from arp record to packet >>>>> ethernet header. >>>>>=20 >>>>> It seems that acquiring lle lock for fast path (main traffic flow) = is >>>>> not necessary even with >>>>> current code. >>>>>=20 >>>>> My tests shows ~10% improvement with this patch applied. >>>>>=20 >>>>> If nobody objects I plan to commit this change at the end of next = week. >>>>=20 >>>> This is risky and prone to race conditions. The copy of the MAC = address >>>> should be done while the table read lock is held to protect against = the >>> It is done exactly as you say: table read lock is held. >>=20 >> How do you protect from entry update if i've a ref to the entry ? >> You can end up doing bcopy of a partial mac address. > I see no problems in copying incorrect mac address in that case: > if host mac address id updated, this is, most likely, another host, = and several packets being lost changes nothing. Sending packet to a bogus mac address is not really nothing :) >=20 > However, there can be some realistic scenario where this can be the = case (L2 load balancing/failover). I'll update in_arpinput() to do lle = removal/insertion in that case. >=20 >> la_preempt modification is also write access to an unlocked = structure. > This one changes nothing: > current code does this under _read_ lock. Under the table lock not the entry lock ? Table lock is here to protect the table if I've understood the code = correctly. If i get an exclusive reference to the entry you will end up reading and = writing to the entry without any lock. >=20 >>=20 >>=20 >>>=20 >>>> entry going away. You can either return with table lock held and = drop >>>> it after the copy, or you could a modified lookup function that = takes a >>>> pointer for the copy destination, do the copy with the read lock, = and then >>>> return. If no entry is found an error is returned and obviously no = copy >>>> is done. >>>>=20 >>>=20 >>>=20 >>> -- >>> WBR, Alexander >>>=20 >>>=20 >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" >>=20 >>=20 >=20 From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 11:21:06 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 43FFA2A8; Fri, 9 Nov 2012 11:21:06 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from mail.ipfw.ru (unknown [IPv6:2a01:4f8:120:6141::2]) by mx1.freebsd.org (Postfix) with ESMTP id B2DDB8FC0C; Fri, 9 Nov 2012 11:21:05 +0000 (UTC) Received: from dhcp170-36-red.yandex.net ([95.108.170.36]) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1TWmhM-000DAD-9a; Fri, 09 Nov 2012 15:24:32 +0400 Message-ID: <509CE6A2.2040200@FreeBSD.org> Date: Fri, 09 Nov 2012 15:18:58 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120627 Thunderbird/13.0.1 MIME-Version: 1.0 To: Fabien Thomas Subject: Re: [patch] reducing arp locking References: <509AEDAC.10002@FreeBSD.org> <509B884F.7040106@networx.ch> <509B88B1.3070905@FreeBSD.org> <49EE4F42-6162-40F4-9DE0-1ACA1289B225@netasq.com> <509CC776.9010200@FreeBSD.org> <37E1F76F-D951-4B36-AF00-039DA1CC5CF3@netasq.com> In-Reply-To: <37E1F76F-D951-4B36-AF00-039DA1CC5CF3@netasq.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net@FreeBSD.org, Andre Oppermann , freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 11:21:06 -0000 On 09.11.2012 13:59, Fabien Thomas wrote: > > Le 9 nov. 2012 à 10:05, Alexander V. Chernikov a écrit : > >> On 09.11.2012 12:51, Fabien Thomas wrote: >>> >>> Le 8 nov. 2012 à 11:25, Alexander V. Chernikov a écrit : >>> >>>> On 08.11.2012 14:24, Andre Oppermann wrote: >>>>> On 08.11.2012 00:24, Alexander V. Chernikov wrote: >>>>>> Hello list! >>>>>> >>>>>> Currently we need to acquire 2 read locks to perform simple 6-byte >>>>>> copying from arp record to packet >>>>>> ethernet header. >>>>>> >>>>>> It seems that acquiring lle lock for fast path (main traffic flow) is >>>>>> not necessary even with >>>>>> current code. >>>>>> >>>>>> My tests shows ~10% improvement with this patch applied. >>>>>> >>>>>> If nobody objects I plan to commit this change at the end of next week. >>>>> >>>>> This is risky and prone to race conditions. The copy of the MAC address >>>>> should be done while the table read lock is held to protect against the >>>> It is done exactly as you say: table read lock is held. >>> >>> How do you protect from entry update if i've a ref to the entry ? >>> You can end up doing bcopy of a partial mac address. >> I see no problems in copying incorrect mac address in that case: >> if host mac address id updated, this is, most likely, another host, and several packets being lost changes nothing. > > Sending packet to a bogus mac address is not really nothing :) > >> >> However, there can be some realistic scenario where this can be the case (L2 load balancing/failover). I'll update in_arpinput() to do lle removal/insertion in that case. >> >>> la_preempt modification is also write access to an unlocked structure. >> This one changes nothing: >> current code does this under _read_ lock. > > Under the table lock not the entry lock ? lle entry is read-locked while la_preempt is modified. > Table lock is here to protect the table if I've understood the code correctly. Yes. > If i get an exclusive reference to the entry you will end up reading and writing to the entry without any lock. Yes. And the only single drawback in worst case can be sending a bit more packets to right (but probably expired) MAC address. I'm talking about the following: ARP stack is just IP -> 6 bytes mapping, there is no reason to make it unnecessary complicated like rte, with references being held by upper layer stack. It does not contain interface pointer, etc.. We may need to r/w lock entry, but for 'control plane' code only. If one acquired exclusive lock and wants to change STATIC flag to non-static or change lle address - this is simply wrong and has to be handled by acquiring table wlock. Current ARP code has some flaws like handling arp expiration, but this patch doesn't change much here.. > >> >>> >>> >>>> >>>>> entry going away. You can either return with table lock held and drop >>>>> it after the copy, or you could a modified lookup function that takes a >>>>> pointer for the copy destination, do the copy with the read lock, and then >>>>> return. If no entry is found an error is returned and obviously no copy >>>>> is done. >>>>> >>>> >>>> >>>> -- >>>> WBR, Alexander >>>> >>>> >>>> _______________________________________________ >>>> freebsd-hackers@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >>> >>> >> > > -- WBR, Alexander From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 14:03:14 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A5F4AB6E; Fri, 9 Nov 2012 14:03:14 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id 39F128FC0A; Fri, 9 Nov 2012 14:03:13 +0000 (UTC) Received: from [10.2.1.1] (unknown [10.2.1.1]) by work.netasq.com (Postfix) with ESMTPSA id 364CC27052C5; Fri, 9 Nov 2012 15:03:07 +0100 (CET) Subject: Re: [patch] reducing arp locking Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-1 From: Fabien Thomas In-Reply-To: <509CE6A2.2040200@FreeBSD.org> Date: Fri, 9 Nov 2012 15:03:06 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <8169CD67-E444-46FC-A7C8-DD6FB59091E1@netasq.com> References: <509AEDAC.10002@FreeBSD.org> <509B884F.7040106@networx.ch> <509B88B1.3070905@FreeBSD.org> <49EE4F42-6162-40F4-9DE0-1ACA1289B225@netasq.com> <509CC776.9010200@FreeBSD.org> <37E1F76F-D951-4B36-AF00-039DA1CC5CF3@netasq.com> <509CE6A2.2040200@FreeBSD.org> To: Alexander V. Chernikov X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@FreeBSD.org, Andre Oppermann , freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 14:03:14 -0000 Le 9 nov. 2012 =E0 12:18, Alexander V. Chernikov a =E9crit : > On 09.11.2012 13:59, Fabien Thomas wrote: >>=20 >> Le 9 nov. 2012 =E0 10:05, Alexander V. Chernikov a =E9crit : >>=20 >>> On 09.11.2012 12:51, Fabien Thomas wrote: >>>>=20 >>>> Le 8 nov. 2012 =E0 11:25, Alexander V. Chernikov a =E9crit : >>>>=20 >>>>> On 08.11.2012 14:24, Andre Oppermann wrote: >>>>>> On 08.11.2012 00:24, Alexander V. Chernikov wrote: >>>>>>> Hello list! >>>>>>>=20 >>>>>>> Currently we need to acquire 2 read locks to perform simple = 6-byte >>>>>>> copying from arp record to packet >>>>>>> ethernet header. >>>>>>>=20 >>>>>>> It seems that acquiring lle lock for fast path (main traffic = flow) is >>>>>>> not necessary even with >>>>>>> current code. >>>>>>>=20 >>>>>>> My tests shows ~10% improvement with this patch applied. >>>>>>>=20 >>>>>>> If nobody objects I plan to commit this change at the end of = next week. >>>>>>=20 >>>>>> This is risky and prone to race conditions. The copy of the MAC = address >>>>>> should be done while the table read lock is held to protect = against the >>>>> It is done exactly as you say: table read lock is held. >>>>=20 >>>> How do you protect from entry update if i've a ref to the entry ? >>>> You can end up doing bcopy of a partial mac address. >>> I see no problems in copying incorrect mac address in that case: >>> if host mac address id updated, this is, most likely, another host, = and several packets being lost changes nothing. >>=20 >> Sending packet to a bogus mac address is not really nothing :) >>=20 >>>=20 >>> However, there can be some realistic scenario where this can be the = case (L2 load balancing/failover). I'll update in_arpinput() to do lle = removal/insertion in that case. >>>=20 >>>> la_preempt modification is also write access to an unlocked = structure. >>> This one changes nothing: >>> current code does this under _read_ lock. >>=20 >> Under the table lock not the entry lock ? > lle entry is read-locked while la_preempt is modified. >=20 >> Table lock is here to protect the table if I've understood the code = correctly. > Yes. >> If i get an exclusive reference to the entry you will end up reading = and writing to the entry without any lock. > Yes. And the only single drawback in worst case can be sending a bit = more packets to right (but probably expired) MAC address. Or partial copy of the new mac. >=20 > I'm talking about the following: > ARP stack is just IP -> 6 bytes mapping, there is no reason to make it = unnecessary complicated like rte, with references being held by upper = layer stack. It does not contain interface pointer, etc.. >=20 > We may need to r/w lock entry, but for 'control plane' code only. > If one acquired exclusive lock and wants to change STATIC flag to = non-static or change lle address - this is simply wrong and has to be = handled by acquiring table wlock. >=20 > Current ARP code has some flaws like handling arp expiration, but this = patch doesn't change much here.. In in_arpinput only exclusive access to the entry is taken during the = update no IF_AFDATA_LOCK that's why i was surprised. ; >=20 >>=20 >>>=20 >>>>=20 >>>>=20 >>>>>=20 >>>>>> entry going away. You can either return with table lock held and = drop >>>>>> it after the copy, or you could a modified lookup function that = takes a >>>>>> pointer for the copy destination, do the copy with the read lock, = and then >>>>>> return. If no entry is found an error is returned and obviously = no copy >>>>>> is done. >>>>>>=20 >>>>>=20 >>>>>=20 >>>>> -- >>>>> WBR, Alexander >>>>>=20 >>>>>=20 >>>>> _______________________________________________ >>>>> freebsd-hackers@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>>> To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" >>>>=20 >>>>=20 >>>=20 >>=20 >>=20 >=20 >=20 > --=20 > WBR, Alexander >=20 >=20 From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 18:13:52 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 664) id 50EAC5CD; Fri, 9 Nov 2012 18:13:52 +0000 (UTC) Date: Fri, 9 Nov 2012 10:13:51 -0800 From: David O'Brien To: Alfred Perlstein Subject: Re: make -jN buildworld on < 512MB ram Message-ID: <20121109181351.GE9916@dragon.NUXI.org> Mail-Followup-To: obrien@freebsd.org, Alfred Perlstein , hackers@freebsd.org References: <509182DA.8070303@mu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <509182DA.8070303@mu.org> X-Operating-System: FreeBSD 10.0-CURRENT X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.20 (2009-06-14) Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: obrien@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 18:13:52 -0000 On Wed, Oct 31, 2012 at 12:58:18PM -0700, Alfred Perlstein wrote: > It seems like the new compiler likes to get up to ~200+MB resident when > building some basic things in our tree. > Unfortunately this causes smaller machines (VMs) to take days because of > swap thrashing. You could try reducing data seg size (ulimit -d) to something matching "RAM" size. GCC has two parameters to try to keep its memory usage "reasonable": http://gcc.gnu.org/onlinedocs/gcc-4.2.4/gcc/Optimize-Options.html 'ggc-min-expand' and 'ggc-min-heapsize'. ggc-min-expand ... If getrlimit is available, the notion of "RAM" is the smallest of actual RAM and RLIMIT_DATA ggc-min-heapsize The default is the smaller of RAM/8, RLIMIT_RSS, or a limit which tries to ensure that RLIMIT_DATA or RLIMIT_AS are not exceeded I don't know if Clang/LLVM has something similar or not. -- -- David (obrien@FreeBSD.org) From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 19:11:13 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4B07FCA3 for ; Fri, 9 Nov 2012 19:11:13 +0000 (UTC) (envelope-from Steven.Sears@netapp.com) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by mx1.freebsd.org (Postfix) with ESMTP id 244DB8FC12 for ; Fri, 9 Nov 2012 19:11:12 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.80,747,1344236400"; d="scan'208";a="708712352" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 09 Nov 2012 11:11:12 -0800 Received: from vmwexceht02-prd.hq.netapp.com (vmwexceht02-prd.hq.netapp.com [10.106.76.240]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id qA9JBCbn021597 for ; Fri, 9 Nov 2012 11:11:12 -0800 (PST) Received: from VMWEXCEHT06-PRD.hq.netapp.com (10.106.77.104) by vmwexceht02-prd.hq.netapp.com (10.106.76.240) with Microsoft SMTP Server (TLS) id 14.2.318.1; Fri, 9 Nov 2012 11:11:11 -0800 Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.216]) by vmwexceht06-prd.hq.netapp.com ([10.106.77.104]) with mapi id 14.02.0318.001; Fri, 9 Nov 2012 11:11:11 -0800 From: "Sears, Steven" To: "freebsd-hackers@freebsd.org" Subject: Memory reserves or lack thereof Thread-Topic: Memory reserves or lack thereof Thread-Index: AQHNvq36DpjcaGVTo0u2vjVo8DKzOA== Date: Fri, 9 Nov 2012 19:10:04 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.51] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 09 Nov 2012 19:22:01 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 19:11:13 -0000 I have a memory subsystem design question that I'm hoping someone can answe= r. I've been looking at a machine that is completely out of memory, as in v_free_count =3D 0,=20 v_cache_count =3D 0,=20 I wondered how a machine could completely run out of memory like this, espe= cially after finding a lack of interrupt storms or other pathologies that w= ould tend to overcommit memory. So I started investigating. Most allocators come down to vm_page_alloc(), which has this guard: if ((curproc =3D=3D pageproc) && (page_req !=3D VM_ALLOC_INTERRUPT)) { page_req =3D VM_ALLOC_SYSTEM; }; if (cnt.v_free_count + cnt.v_cache_count > cnt.v_free_reserved || (page_req =3D=3D VM_ALLOC_SYSTEM &&=20 cnt.v_free_count + cnt.v_cache_count > cnt.v_interrupt_free_min) || (page_req =3D=3D VM_ALLOC_INTERRUPT && cnt.v_free_count + cnt.v_cache_count > 0)) { The key observation is if VM_ALLOC_INTERRUPT is set, it will allocate every= last page. >From the name one might expect VM_ALLOC_INTERRUPT to be somewhat rare, perh= aps only used from interrupt threads. Not so, see kmem_malloc() or uma_smal= l_alloc() which both contain this mapping: if ((flags & (M_NOWAIT|M_USE_RESERVE)) =3D=3D M_NOWAIT) pflags =3D VM_ALLOC_INTERRUPT | VM_ALLOC_WIRED; else pflags =3D VM_ALLOC_SYSTEM | VM_ALLOC_WIRED; Note that M_USE_RESERVE has been deprecated and is used in just a handful o= f places. Also note that lots of code paths come through these routines. What this means is essentially _any_ allocation using M_NOWAIT will bypass = whatever reserves have been held back and will take every last page availab= le. There is no documentation stating M_NOWAIT has this side effect of essentia= lly being privileged, so any innocuous piece of code that can't block will = use it. And of course M_NOWAIT is literally used all over. It looks to me like the design goal of the BSD allocators is on recovery; i= t will give all pages away knowing it can recover. Am I missing anything? I would have expected some small number of pages to = be held in reserve just in case. And I didn't expect M_NOWAIT to be a sort = of back door for grabbing memory. Thanks, -Steve From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 20:34:45 2012 Return-Path: Delivered-To: hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9FF2D26C for ; Fri, 9 Nov 2012 20:34:45 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og106.obsmtp.com (exprod7og106.obsmtp.com [64.18.2.165]) by mx1.freebsd.org (Postfix) with ESMTP id 1CEE88FC08 for ; Fri, 9 Nov 2012 20:34:41 +0000 (UTC) Received: from P-EMHUB01-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob106.postini.com ([64.18.6.12]) with SMTP ID DSNKUJ1o2i5AyJLGcKVBAnKS2iy+B19ZDYuJ@postini.com; Fri, 09 Nov 2012 12:34:45 PST Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB01-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Fri, 9 Nov 2012 12:32:43 -0800 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id qA9KWf358752; Fri, 9 Nov 2012 12:32:42 -0800 (PST) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id C5DD058094; Fri, 9 Nov 2012 12:32:40 -0800 (PST) To: Subject: Re: Fwd: Re: make -jN buildworld on < 512MB ram In-Reply-To: <20121109175641.GC9916@dragon.NUXI.org> References: <20121109175641.GC9916@dragon.NUXI.org> Comments: In-reply-to: "David O'Brien" message dated "Fri, 09 Nov 2012 09:56:41 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Fri, 9 Nov 2012 12:32:40 -0800 Message-ID: <20121109203240.C5DD058094@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-Mailman-Approved-At: Fri, 09 Nov 2012 22:29:34 +0000 Cc: sjg@juniper.net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 20:34:45 -0000 >Ah, but make(1) can delay spawning any new processes when it knows its >children are paging. Sounds good - but I doubt the complexity it adds would really be worth it. If the real problem is the C++ compiler or other large tool needing sufficient resources that you have to ensure invocations are sequenced/throttled, then the obvious solution would seem to be to invoke it via a proxy that can do the the needed throttling on a global (system, user, or whatever basis seems to make sense). Think of make's invocation of ${CXX} as actually just submitting a job to the CXX service, which might only be allowing one instance of C++ to run at a time. From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 22:33:00 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3D8CDB91; Fri, 9 Nov 2012 22:33:00 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id CE6708FC08; Fri, 9 Nov 2012 22:32:59 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so5685327oag.13 for ; Fri, 09 Nov 2012 14:32:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+ov+6zUAOqvpVQ9XUPsO/wIJ8nB2NFOKKFfu/qNYFnA=; b=YUE6hMOxS3K9WXHsHb1A3BtQP6B6HqZsXvGjsQJV6XMLX2miGvvpxHri1sqUAVRGiR S4xC9S6CCSUV1T3gQM3EIGnDQWZlTy88DWCb2dPvxdMnN4olqGsOTnuQC9C0YrwSebva 8/ALPjaKdw5JrVmk1VcbWOIMcpbbmwTdAHgFeRUBU5Lt6BlCxn17Ue+2R78TEU8g3tfc jeo0LhfHXAwW5jm7JqjIApIBkc/HN0jJV2WfsBLy+QYQHRFMTWiuil1zdpeYy1/57bqW WY6I5ElHp9FPCaaDQ1Ev9uHFWDYUbIGiPe79QYXvvKVGZ1+rB0dW8MlYAixcJ0GjrQdy K1uA== MIME-Version: 1.0 Received: by 10.60.32.171 with SMTP id k11mr9003765oei.113.1352500373205; Fri, 09 Nov 2012 14:32:53 -0800 (PST) Received: by 10.182.133.40 with HTTP; Fri, 9 Nov 2012 14:32:53 -0800 (PST) In-Reply-To: <509CC1F6.1010308@freebsd.org> References: <31C904E6-F230-4187-AE32-F9A7B1A7C38E@freebsd.org> <509CC1F6.1010308@freebsd.org> Date: Sat, 10 Nov 2012 00:32:53 +0200 Message-ID: Subject: Re: FreeBSD on RaspberryPi From: Sami Halabi To: Stefan Esser Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-hackers@freebsd.org, Tim Kientzle X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 22:33:00 -0000 Hi, are there any plans to do images that support the hdmi output ? what about analog sound vs the hdmi? can the images be taken to the gui stage , ie xbmc or any other multimedia screen ? Sami On Fri, Nov 9, 2012 at 10:42 AM, Stefan Esser wrote: > Am 09.11.2012 05:44, schrieb Tim Kientzle: > >> On Wed, Nov 7, 2012 at 6:01 PM, Tim Kientzle > wrote: > >> WARNING: This is still highly experimental and by no > >> means ready for "production use", ... > >> > >> To boot FreeBSD on your RaspberryPi, you'll need: > >> 1) A RaspberryPi. > >> 2) A serial cable similar to this one: www.adafruit.com/products/954 > > > > > > On Nov 8, 2012, at 9:13 AM, Sami Halabi wrote: > >> > >> why the console cable is needed ? > >> > > As far as I can tell, the code in FreeBSD-CURRENT > > does not yet support the video out. So you need > > a serial console cable to interact with it. > > All it takes to get the framebuffer working is that the hash chars are > removed. I.e. the following works: > > device sc > device kbdmux > options SC_DFLT_FONT # compile font in > makeoptions SC_DFLT_FONT=cp437 > > > You might be able to interact via SSH but > > that requires a little bit more setup (a root > > password needs to be set and you need to > > edit /etc/ssh/sshd_config to allow root logins). > > I used SSH, and the framebuffer helps to see how far the boot process > has come. It takes about 60 seconds to generate the SSH host keys, for > example. > > [The following points are not specific to the R-PI, and I'm sure you > know them, but I list them for others that may want to use their R-PI > without serial console.] > > In order to use SSH I modified sshd_config to accept direct root logins. > The root password must be set (best method: "vipw -d /mnt/etc", else > you must remember to invoke "pwd_mkdb -d /mnt/etc" when you are done). > > The host name and IP address should be set in rc.conf (or assigned via > DHCP). > > If you do not want to enable direct root login, then a non-privileged > account in group wheel is required to be able to "su" to root. > > That's all I remember ... > > I used the build script from "http://kernelnomicon.org/?p=164" with > one slight modification (tar -x ... --no-same-owner ...). > > My R-PI kernel contains MSDOSFS and NFS client support to allow it > to mount its boot partition and NFS exported /usr/src, /usr/obj, > /usr/ports and /usr/work (where I build ports). Most of them are > R/O mounts. I have not tried to build world on the R-PI (cross > building is so much faster ...). But ports can be build, if a swap > partition is available (e.g. on SD card or via NFS - I did not try > to mount a USB stick, but that might be another option). > > Regards, STefan > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 10 13:20:25 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 125D144C for ; Sat, 10 Nov 2012 13:20:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id A32FC8FC15 for ; Sat, 10 Nov 2012 13:20:24 +0000 (UTC) Received: from tom.home (localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qAADKKP4073948; Sat, 10 Nov 2012 15:20:20 +0200 (EET) (envelope-from kostikbel@gmail.com) X-DKIM: OpenDKIM Filter v2.5.2 kib.kiev.ua qAADKKP4073948 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qAADKJBS073940; Sat, 10 Nov 2012 15:20:19 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 10 Nov 2012 15:20:19 +0200 From: Konstantin Belousov To: "Sears, Steven" Subject: Re: Memory reserves or lack thereof Message-ID: <20121110132019.GP73505@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pc5/sMjAdU99/gPV" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=0.2 required=5.0 tests=ALL_TRUSTED, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Nov 2012 13:20:25 -0000 --pc5/sMjAdU99/gPV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 09, 2012 at 07:10:04PM +0000, Sears, Steven wrote: > I have a memory subsystem design question that I'm hoping someone can ans= wer. >=20 > I've been looking at a machine that is completely out of memory, as in >=20 > v_free_count =3D 0,=20 > v_cache_count =3D 0,=20 >=20 > I wondered how a machine could completely run out of memory like this, es= pecially after finding a lack of interrupt storms or other pathologies that= would tend to overcommit memory. So I started investigating. >=20 > Most allocators come down to vm_page_alloc(), which has this guard: >=20 > if ((curproc =3D=3D pageproc) && (page_req !=3D VM_ALLOC_INTERRUPT)) { > page_req =3D VM_ALLOC_SYSTEM; > }; >=20 > if (cnt.v_free_count + cnt.v_cache_count > cnt.v_free_reserved || > (page_req =3D=3D VM_ALLOC_SYSTEM &&=20 > cnt.v_free_count + cnt.v_cache_count > cnt.v_interrupt_free_min) || > (page_req =3D=3D VM_ALLOC_INTERRUPT && > cnt.v_free_count + cnt.v_cache_count > 0)) { >=20 > The key observation is if VM_ALLOC_INTERRUPT is set, it will allocate eve= ry last page. >=20 > >From the name one might expect VM_ALLOC_INTERRUPT to be somewhat rare, p= erhaps only used from interrupt threads. Not so, see kmem_malloc() or uma_s= mall_alloc() which both contain this mapping: >=20 > if ((flags & (M_NOWAIT|M_USE_RESERVE)) =3D=3D M_NOWAIT) > pflags =3D VM_ALLOC_INTERRUPT | VM_ALLOC_WIRED; > else > pflags =3D VM_ALLOC_SYSTEM | VM_ALLOC_WIRED; >=20 > Note that M_USE_RESERVE has been deprecated and is used in just a handful= of places. Also note that lots of code paths come through these routines. >=20 > What this means is essentially _any_ allocation using M_NOWAIT will bypas= s whatever reserves have been held back and will take every last page avail= able. >=20 > There is no documentation stating M_NOWAIT has this side effect of essent= ially being privileged, so any innocuous piece of code that can't block wil= l use it. And of course M_NOWAIT is literally used all over. >=20 > It looks to me like the design goal of the BSD allocators is on recovery;= it will give all pages away knowing it can recover. >=20 > Am I missing anything? I would have expected some small number of pages t= o be held in reserve just in case. And I didn't expect M_NOWAIT to be a sor= t of back door for grabbing memory. >=20 Your analysis is right, there is nothing to add or correct. This is the reason to strongly prefer M_WAITOK. --pc5/sMjAdU99/gPV Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlCeVJMACgkQC3+MBN1Mb4hR/gCbB/O8BhKBT5X1R0N4qgE2j3rN psMAn2+n5ZpjGJpiPsf/zPXLnr3B4QuO =6RHi -----END PGP SIGNATURE----- --pc5/sMjAdU99/gPV-- From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 10 20:07:25 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6D83C6BC; Sat, 10 Nov 2012 20:07:25 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 3F0B28FC08; Sat, 10 Nov 2012 20:07:24 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qAAK7Hh4037959; Sat, 10 Nov 2012 13:07:24 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qAAK75gq019245; Sat, 10 Nov 2012 13:07:05 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Subject: Re: watchdogd, jemalloc, and mlockall From: Ian Lepore To: freebsd-embedded@freebsd.org, freebsd-hackers@freebsd.org In-Reply-To: <1351968635.1120.110.camel@revolution.hippie.lan> References: <1351967919.1120.102.camel@revolution.hippie.lan> <20121103184143.GC73505@kib.kiev.ua> <1351968635.1120.110.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Sat, 10 Nov 2012 13:07:05 -0700 Message-ID: <1352578025.17290.123.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Jason Evans X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Nov 2012 20:07:25 -0000 On Sat, 2012-11-03 at 12:50 -0600, Ian Lepore wrote: > On Sat, 2012-11-03 at 20:41 +0200, Konstantin Belousov wrote: > > On Sat, Nov 03, 2012 at 12:38:39PM -0600, Ian Lepore wrote: > > > In an attempt to un-hijack the thread about memory usage increase > > > between 6.4 and 9.x, I'm starting a new thread here related to my recent > > > discovery that watchdogd uses a lot more memory since it began using > > > mlockall(2). > > > > > > I tried statically linking watchdogd and it made a small difference in > > > RSS, presumably because it doesn't wire down all of libc and libm. > > > > > > VSZ RSS > > > 10236 10164 Dynamic > > > 8624 8636 Static > > > > > > Those numbers are from ps -u on an arm platform. I just updated the PR > > > (bin/173332) with some procstat -v output comparing with/without > > > mlockall(). > > > > > > It appears that the bulk of the new RSS bloat comes from jemalloc > > > allocating vmspace in 8MB chunks. With mlockall(MCL_FUTURE) in effect > > > that leads to wiring 8MB to satisfy what probably amounts to a few > > > hundred bytes of malloc'd memory. > > > > > > It would probably also be a good idea to remove the floating point from > > > watchdogd to avoid wiring all of libm. The floating point is used just > > > to turn the timeout-in-seconds into a power-of-two-nanoseconds value. > > > There's probably a reasonably efficient way to do that without calling > > > log(), considering that it only happens once at program startup. > > > > No, I propose to add a switch to turn on/off the mlockall() call. > > I have no opinion on the default value of the suggested switch. > > In a patch I submitted along with the PR, I added code to query the > vm.swap_enabled sysctl and only call mlockall() when swapping is > enabled. > > Nobody yet has said anything about what seems to me to be the real > problem here: jemalloc grabs 8MB at a time even if you only need to > malloc a few bytes, and there appears to be no way to control that > behavior. Or maybe there's a knob in there that didn't jump out at me > on a quick glance through the header files. I finally found some time to pursue this further. A small correction to what I said earlier: it appears that jemalloc allocates chunks of 4MB at a time, not 8, but it also appears that it allocates at least 2 chunks so the net effect is an 8MB default minimum allocation. I played with the jemalloc tuning option lg_chunk and with static versus dynamic linking, and came up with the numbers below, which were generated by ps -u on an ARM-based system with 64MB running -current from a couple weeks ago, but with the recent patch to watchdogd to eliminate the need for libm. I used "lg_chunk:14" (16K chunks), the smallest value it would allow on this platform. For comparison I also include the numbers from a FreeBSD 8.2 ARM system (which would be dynamic linked and untuned, and also without any mlockall() calls). Link malloc %MEM VSZ RSS ------------------------------------- dynamic untuned 15.3 10040 9996 static untuned 13.2 8624 8636 dynamic tuned 2.8 1880 1836 static tuned 0.8 480 492 [ freebsd 8.2 ] 1.1 1752 748 So it appears that using jemalloc's tuning in a daemon that uses mlockall(2) is a big win, especially if the daemon doesn't do much memory allocation (watchdogd allocates 2 things, 4k and 1280 bytes; if you use -e it also strdup()s the command string). It also seems that providing a build-time knob to control static linking would be valuable on platforms that are very memory limited and can't benefit from having all of libc wired. I haven't attached a patch because there appears to be no good way to actually achieve this in a platform-agnostic way. The jemalloc code enforces the lower range of the lg_chunk tuning value to be tied to the page size of the platform, and it rejects out of range values without changing the tuning. The code that works on an ARM with 4K page size, const char *malloc_conf = "lg_chunk:14"; would fail on a system that had bigger pages. The tuning must be specified with a compile-time constant like that, because it has to be tuned before the first allocation, which apparently happens before main() is entered. It would be nice if jemalloc would clip the tuning to the lowest legal value instead of rejecting it, especially since the lowest legal value is calculated based not only on page size but on the value of other configurable values. There's another potential solution, but it strikes me as rather inelegant... jemalloc can also be tuned with the MALLOC_CONF env var. With the right rc-fu we could provide something like a watchdogd_memtune variable that you could set and watchdogd would be invoked with MALLOC_CONF set to that in the environment. It still couldn't be set to a default value that was good for all platforms. It would also get passed through environment inheritence to any "-e whatever" command run by watchdogd, which isn't necessarily appropriate. I'm cc'ing Jason in case he can offer some advice about a better way to achieve this tuning, because I'm sure my quick read-through of the manpage this morning missed important details and implications. -- Ian