Clusteradm

From owner-svn-doc-head@freebsd.org Sat Apr 16 23:20:52 2016 Return-Path: Delivered-To: svn-doc-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 719F1B10D97; Sat, 16 Apr 2016 23:20:52 +0000 (UTC) (envelope-from wblock@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34FAD1B99; Sat, 16 Apr 2016 23:20:52 +0000 (UTC) (envelope-from wblock@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u3GNKpme070658; Sat, 16 Apr 2016 23:20:51 GMT (envelope-from wblock@FreeBSD.org) Received: (from wblock@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u3GNKp7q070657; Sat, 16 Apr 2016 23:20:51 GMT (envelope-from wblock@FreeBSD.org) Message-Id: <201604162320.u3GNKp7q070657@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: wblock set sender to wblock@FreeBSD.org using -f From: Warren Block Date: Sat, 16 Apr 2016 23:20:51 +0000 (UTC) To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r48656 - head/en_US.ISO8859-1/htdocs/news/status X-SVN-Group: doc-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-doc-head@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: SVN commit messages for the doc tree for head List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Apr 2016 23:20:52 -0000 Author: wblock Date: Sat Apr 16 23:20:51 2016 New Revision: 48656 URL: https://svnweb.freebsd.org/changeset/doc/48656 Log: General editing and markup additions. Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2016-01-2016-03.xml Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2016-01-2016-03.xml ============================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2016-01-2016-03.xml Sat Apr 16 19:50:04 2016 (r48655) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2016-01-2016-03.xml Sat Apr 16 23:20:51 2016 (r48656) @@ -350,7 +350,7 @@ Jared McNeill - jmcneill@freebsd.org + jmcneill@FreeBSD.org @@ -770,14 +770,14 @@ -

POSIX specifies several kinds of pthread locks, for this - report the private and process-shared variants are considered. +

POSIX specifies several kinds of pthread locks. For this + report, the private and process-shared variants are considered. Private locks can be used only by the threads of the same process, which share the address space. Process-shared locks can be used by threads from any process, assuming the process can map the lock memory into its address space.

Our libthr, the library implementing the POSIX threads and +

Our libthr, the library implementing the POSIX threads and locking operations, uses a pointer as the internal representation behind a lock. The pointer contains the address of the actual structure carrying the lock. This has @@ -787,11 +787,11 @@ distinct address spaces.

A common opinion was that we have no choice but to break the - libthr Application Binary Interface (ABI) by changing the lock + libthr Application Binary Interface (ABI) by changing the lock types to be the actual lock structures (and padding for future ABI extension). This is very painful for users, as our - previous experience with non-versioned libc and libc_r - shown.

+ previous experience with non-versioned libc and libc_r + has shown.

Instead, I proposed and implemented a scheme where process-shared locks can be implemented without breaking the @@ -799,13 +799,13 @@ hash of the shared memory objects (off-pages), which carry the real lock structures.

New umtx operations to create or look up the shared object, - by the memory key, were added. Libthr is modified to lookup +

New umtx operations to create or look up the shared object, + by the memory key were added. libthr is modified to look up the object and use it for shared locks, instead of using - malloc() as for private locks.

+ malloc() as for private locks.

The pointer value in the user-visible lock type contains a - canary for shared locks. Libthr detects the canary and + canary for shared locks. libthr detects the canary and switches into the shared-lock mode.

The proposal of inlining the lock structures, besides the @@ -814,16 +814,16 @@ important advantage over the off-page page approach is that no off-page object needs to be maintained, and the lifecycle of the shared lock naturally finishes with the destruction of the - shared memory, without explicit cleanup. Right now, off-pages - hook into vm object termination to avoid leakage, but long - liviness of the vnode vm object prolonges the off-page + shared memory without explicit cleanup. Right now, off-pages + hook into vm object termination to avoid leakage, but + long-livedness of the vnode vm object prolonges the off-page existence for shared locks backed by files, however unlikely they may be.

Libthr with inlined locks become informally known as libthr2 - project, since the library name better be changed instead of - only bumping the library version. The rtld should ensure that - libthr and libthr2 do not become simultaneously loaded into a +

libthr with inlined locks became informally known as the libthr2 + project, since it is better to change the library name than just + bumping the library version. rtld should ensure that + libthr and libthr2 are not simultaneously loaded into a single address space.

@@ -835,17 +835,17 @@ -

Evaluate and implement libthr2.

Evaluate and implement libthr2.

- Clusteradm + Cluster Admin - clusteradm@freebsd.org + clusteradm@FreeBSD.org @@ -856,7 +856,7 @@

migrated services out of the hosting space in ISC (peter, sbruno)

begun migration of services into RootBSD hosting space +

began migration of services into RootBSD hosting space (peter, sbruno)

collaborated with phabricator admin team to migrate to @@ -1079,7 +1079,7 @@ Justin Hibbits - jhibbits@freebsd.org + jhibbits@FreeBSD.org @@ -1309,7 +1309,7 @@ Hans Petter Selasky - hselasky@freebsd.org + hselasky@FreeBSD.org @@ -1600,20 +1600,22 @@ - Patch home. + Patch home + paxtest.log + fedora.log

I wrote a small and straightforward yet feature-packed patch - to implement ASLR for &os; available for broader testing.

+ to implement ASLR for &os; which is now available for broader testing.

With this change, randomization is applied to all non-fixed mappings. By randomization I mean the base address for the mapping is selected with a guaranteed amount of entropy (bits). If the mapping was requested to be superpage aligned, - the randomization honours the superpage attributes.

+ the randomization honors the superpage attributes.

The randomization is done on a best-effort basis - that is, +

The randomization is done on a best-effort basis. That is, the allocator falls back to a first fit strategy if fragmentation prevents entropy injection. It is trivial to implement a strong mode where failure to guarantee the @@ -1623,43 +1625,43 @@

I have not fine-tuned the amount of entropy injected right now. It is only a quantitive change that will not change the implementation. The current amount is controlled by - aslr_pages_rnd.

+ aslr_pages_rnd.

To not spoil coalescing optimizations, to reduce the page table fragmentation inherent to ASLR, and to keep the - transient superpage promotion for the malloced memory, the + transient superpage promotion for the malloced memory, the locality is implemented for anonymous private mappings, which are automatically grouped until fragmentation kicks in. The initial location for the anon group range is, of course, randomized. After some additional tuning, the measures - appeared to be quite effective. In particular, very - address-space hungry build of PyPy 5.0 on i386 successfully + appeared to be quite effective. In particular, a very + address-space-hungry build of PyPy 5.0 on i386 successfully finished with the most aggressive functionality of the patch activated.

The default mode keeps the sbrk area unpopulated by other +

The default mode keeps the sbrk area unpopulated by other mappings, but this can be turned off, which gives much more breathing bits on the small AS architectures (funny that - 32bits is considered small). This is tied with the question + 32 bits is considered small). This is tied with the question of following an application's hint about the mmap(2) base address. Testing shows that ignoring the hint does not affect the function of common applications, but I would expect - more demanding code could break. By default sbrk is preserved - and mmap hints are satisfied, which can be changed by using - the kern.elf{32,64}.aslr_care_sbrk sysctl (currently enabled + more demanding code could break. By default sbrk is preserved + and mmap hints are satisfied, which can be changed by using + the kern.elf{32,64}.aslr_care_sbrk sysctl (currently enabled by default for wider testing).

Stack gap, W^X, shared page randomization, KASLR and other techniques are explicitly out of scope of this work.

The paxtest results for the run with the previous version 5 - of the patch applied and aggresively tuned can be seen at the - https://www.kib.kiev.ua/kib/aslr/paxtest.log . For +

The paxtest results for the run with the previous version 5 + of the patch applied and aggressively tuned can be seen at + paxtest.log. For comparison, the run on Fedora 23 on the same machine is at - https://www.kib.kiev.ua/kib/aslr/fedora.log .

+ fedora.log.

ASLR is enabled on per-ABI basis, and currently it is only - enabled on native i386 and amd64 (including compat 32bit) and +

ASLR is enabled on a per-ABI basis, and currently is only + enabled on native i386 and amd64 (including compat 32-bit) and ARMv6 ABIs. I expect to test and enable ASLR for arm64 as well, later.

@@ -1667,7 +1669,7 @@ I have not provided a userspace wrapper around the syscall. In fact, the most reasonable control needed is per-image and not per-process, but we have no tradition to put the - kernel-read attributes into the extattrs of binary, so I am + kernel-read attributes into the extattrs of a binary, so I am still pondering that part and this also explains the non-written tool.

@@ -1680,7 +1682,7 @@ Foundation for directing me.

Bartek Rutkowski tested PyPy builds on i386, and David Naylor - helped with the port which was at point of turbulence and + helped with the port which was at the point of turbulence and upgrade during the work.