ptnet Driver and bhyve Device Model

From owner-svn-doc-head@freebsd.org Fri Oct 28 18:10:41 2016 Return-Path: Delivered-To: svn-doc-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 84A77C24683; Fri, 28 Oct 2016 18:10:41 +0000 (UTC) (envelope-from wblock@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 531CABB4; Fri, 28 Oct 2016 18:10:41 +0000 (UTC) (envelope-from wblock@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u9SIAe12060726; Fri, 28 Oct 2016 18:10:40 GMT (envelope-from wblock@FreeBSD.org) Received: (from wblock@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u9SIAecv060725; Fri, 28 Oct 2016 18:10:40 GMT (envelope-from wblock@FreeBSD.org) Message-Id: <201610281810.u9SIAecv060725@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: wblock set sender to wblock@FreeBSD.org using -f From: Warren Block Date: Fri, 28 Oct 2016 18:10:40 +0000 (UTC) To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r49603 - head/en_US.ISO8859-1/htdocs/news/status X-SVN-Group: doc-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-doc-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the doc tree for head List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Oct 2016 18:10:41 -0000 Author: wblock Date: Fri Oct 28 18:10:40 2016 New Revision: 49603 URL: https://svnweb.freebsd.org/changeset/doc/49603 Log: Grammar and spelling changes, comma and semicolon reductions, run-on sentences debigulated, Britishisms Muricanized. Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2016-07-2016-09.xml Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2016-07-2016-09.xml ============================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2016-07-2016-09.xml Fri Oct 28 17:34:19 2016 (r49602) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2016-07-2016-09.xml Fri Oct 28 18:10:40 2016 (r49603) @@ -110,10 +110,10 @@

Currently, &os; is well proven as a base for routers (pfSense, OPNSense, BSDRP) and NAS (FreeNAS, - zfsGuru, Nas4Free). However, + zfsGuru, NAS4Free). However, &os;-based solutions are almost completely absent in the virtualization area, and ClonOS is one of the - attempts to change it. + attempts to change that.

ClonOS is a new free open-source &os;-based platform for virtual @@ -138,7 +138,7 @@ management)

additional features such as go-micro services (obtaining - VMs, resizing disks and so on)

+ VMs, resizing disks, and so on) @@ -147,7 +147,7 @@ regard we are interested in finding more people and companies who used &os; in hosting tasks. In addition, it could be great to work with the developers of existing NAS solutions (zfsGuru, - Nas4Free) + NAS4Free). @@ -358,7 +358,7 @@ - ptnet Driver and bhyve Device Model + <tt>ptnet</tt> Driver and <tt>bhyve</tt> Device Model @@ -491,7 +491,7 @@ lightweight, while still being visually appealing and easy to use.

During this quarter, the team has kept the following applications +

During this quarter, the team has kept these applications up-to-date:

This project was started during Google Summer of Code this year. The aim was to create a library which can convert the audit trail files in Linux Audit format or the format used by Windows to the BSM - format (the format &os; uses for its audit logs). Apart from that, + format used by &os; for its audit logs. Apart from that, I wanted to create a simple command-line tool and extend auditdistd so that it is possible to send non-BSM logs to - auditdistd over a secure connection and save those audit + it over a secure connection and save those audit logs on disk, preferably in the BSM format.

So far, it is possible to reasonably convert some of the most @@ -597,8 +597,8 @@ command-line tool is usable but not perfect.

The present work focuses on configuring the secure TLS connection - between CentOS and auditdistd. I've already tried using - rsyslogd but wasn't able to make it work.

auditdistd

@@ -613,7 +613,7 @@ Configure auditdistd to be able to communicate with some software on CentOS over TLS in order to receive audit logs. I - wasn't able to come up with a simple solution for that. + was not able to come up with a simple solution for that. Additional open tasks are listed on the Wiki page and in the TODO file in the root directory of the project. @@ -635,7 +635,7 @@

Non-Transparent Bridges allow creation of memory windows between - different systems, using the regular PCIe links of CPUs as a + different systems using the regular PCIe links of CPUs as a transport. During the last quarter, the NTB subsystem gained a significant set of improvements and fixes:

@@ -752,7 +752,7 @@ has been tested and improved in order to gain production quality. Most of this effort has been invested in development and benchmarking of the on-chip Gigabit Ethernet (NETA) functionality. - Numerous bug fixes, as well as some new features, have been + Numerous bug fixes and some new features have been introduced.

Work completed this quarter includes:

@@ -963,7 +963,7 @@ fixes or functionality in the trees of NetBSD, &os;, OpenBSD, and DragonFly BSD.

One thing which became obvious very quickly, was the +

One thing which became obvious very quickly was the inconsistency between operating systems about where and/or which version a utility originated in, despite our common heritage.

@@ -971,9 +971,9 @@ details in pages which already had a history section and making patches for those which did not.

From there, changes were propogated out to NetBSD, OpenBSD and +

From there, changes were propogated out to NetBSD, OpenBSD, and Dragonfly BSD where applicable (not all utilities originated from - the same source or implimentation, for example).

+ the same source or implementation, for example).

This was a good exercise in:

@@ -986,7 +986,7 @@

Becoming familiar with the locations where things are documented and with external sources of historical information, - such as the BSD Family Tree which is included in the &os; base + such as the BSD Family Tree included in the &os; base system, and projects like The UNIX Heritage Society and the manual library on cat-v.org which @@ -999,10 +999,10 @@ Cover the remaining manuals for userland utilities, and maybe - expand onto library and syscall APIs, though I say that without - estimating the feasibility — components originating from a - closed-source operating system are tricky to document the history - of, since older versions are not always available. + expand into library and syscall APIs, though I say that without + estimating the feasibility. The history of components originating from a + closed-source operating system is tricky to document, + since older versions are not always available. @@ -1032,8 +1032,8 @@ -

&os; provides an API for guest OSes to access shared folders on - the host OS so that the kernel driver can expose them to the +

&os; provides an API for guest operating systems to access shared folders on + the host so that the kernel driver can expose them to the guest's userland. This project aims to add such functionality to the VirtualBox Guest Additions driver.

@@ -1046,14 +1046,14 @@ Finish the missing pieces. - implement proper locking. + Implement proper locking. - general clean-up and bugfixes. + General clean-up and bugfixes. - evdev Support + <tt>evdev</tt> Support @@ -1081,7 +1081,7 @@

evdev is a portable, API-compatible implementation of the Linux /dev/input/eventX interface. It covers a wide - variaty of input devices like keyboards, mice, and touchscreens + variety of input devices like keyboards, mice, and touchscreens (with multitouch support), and support for it is implemented in a lot of existing userland components like Qt, libinput, and tslib.

@@ -1093,7 +1093,7 @@ also added for TI's AM33xx touchstreen controller (the popular BeagleBone is based on the AM33xx) and the official touschreen for the Raspberry Pi. Multitouch support for the Raspberry Pi was - successfully demoed using the lastest Qt development branch.

+ successfully demonstarted using the latest Qt development branch.

@@ -1134,11 +1134,11 @@ -

Transparent superpage support has been added. This will allow +

Transparent superpage support has been added. This allows &os; to create 2MiB blocks with a single pagetable and TLB entry. - This has shown a small but significant improvement in the + This shows a small but significant improvement in the buildworld time on ThunderX machines. Superpages have been - enabled in head with and merged to stable/11, but they are + enabled in head and merged to stable/11, but they are disabled by default on stable/11 due to a lack of testing there.

@@ -1230,12 +1230,12 @@ leveldb.

In our development repository, we have done the following +

In our development repository, we have done this work:

The plasma5 branch has been kept up to date with KDE's upstream and - contains ports for Frameworks 5.26.0, Plasma Desktop 5.8.0 and + contains ports for Frameworks 5.26.0, Plasma Desktop 5.8.0, and Applications 16.08.1 (branches/plasma5).

@@ -1256,14 +1256,14 @@

The third quarter started with the handover to the ninth Core team as it took office. With four members returning from the previous core (Baptiste Daroussin, Ed Maste, George Neville-Neil - and Hiroki Sato); one returning member after a term away (John - Baldwin) and four members new to core (Allan Jude, Kris Moore, - Benedict Reuschling and Benno Rice) the new core team represents + and Hiroki Sato), one returning member after a term away (John + Baldwin), and four members new to core (Allan Jude, Kris Moore, + Benedict Reuschling and Benno Rice), the new core team represents just about the ideal balance between experience and fresh blood.

Beyond handing over all of the ongoing business, reviewing - everything on Core's agenda and other routine changeover + everything on Core's agenda, and other routine changeover activities, the first action of the new core was to respond to a query from Craig Rodrigues concerning how hardware supplied to the project through donations to the &os; Foundation was being @@ -1272,7 +1272,7 @@

The Foundation does keep records of what hardware has been supplied over time and has some idea of the original purpose that hardware was provisioned for, but does not track the current usage - of the project's hardware assets. Cluster administration keep + of the project's hardware assets. Cluster administration keeps their own configuration database, but this is not suitable for general publication and covers much more than Foundation supplied equipment. After some discussion it was decided that updated @@ -1327,7 +1327,7 @@ published. Early access to information about vulnerabilities is contingent on their ability to avoid premature disclosure, and without such, they could not have security advisories and - patches ready to go immediately the vulnerability is + patches ready to go immediately when the vulnerability is published.

However, in this case, vulnerabilities were already public and @@ -1343,8 +1343,8 @@

The OpenSSH project has deprecated DSA keys upstream. &os; had kept DSA keys enabled in the later 10.x releases for compatibility reasons, but with the release of 11.0 the time has come to - synchronise again with upstream. Since there are numerous DSA - keys in use in the &os; cluster this has necessitated an + synchronize again with upstream. Since there are numerous DSA + keys in use in the &os; cluster, this necessitated an exercise to get replacement keys installed. Core would like to thank David Wolfskill and the accounts team for handling the surge in key changes with a great deal of aplomb.

@@ -1376,41 +1376,41 @@ existing code correctly interoperated with readers, both kernel- and user-space, giving them lock-less access to the actual data ('timehands') needed to derive the time of day from the - timecounter hardware, in the presence of updaters. But updates + timecounter hardware in the presence of updaters. But updates of the timehands, which are performed by periodic clock - interrupts, the ntpd-driven sys_ntp_adjtime(2) syscall, the - settimeofday(2) syscall, pps sync, and possibly other sources, + interrupts, the ntpd-driven sys_ntp_adjtime(2) syscall, the + settimeofday(2) syscall, pps sync, and possibly other sources, were not coordinated. Moreso, the NTP code was locked by Giant, which did not really serve any purpose.

As result of the work, locking was applied to ensure that any - timehands adjustments are performed by single mutator. An +

As a result of the work, locking was applied to ensure that any + timehands adjustments are performed by a single mutator. An interesting case is the parallel modification of the timehands - from the top of the kernel, for instance the settimeoday(2) + from the top of the kernel, for instance the settimeoday(2) syscall, and a simultaneous clock tick event, where the syscall has already acquired the resources. In this case, it is highly desirable to not block (spin) in the tick handler, and the required adjustments are performed by the already executing call - from top half. There, the typical trylock operation is desired, - which was surprisingly lacking from our spinlock implementation. - mtx_trylock_spin(9) was implemented and is used for this + from the top half. There, the typical trylock operation is desired, + which was surprisingly lacking in our spinlock implementation. + mtx_trylock_spin(9) was implemented and is used for this purpose.

The userspace getttimeofday(2) implementation was enhanced to +

The userspace getttimeofday(2) implementation was enhanced to allow syscall-less operation on machines that use HPET hardwire for timecounters. The HPET algorithm coexists with older RDTSC-based code, allowing dynamic switching of timecounters. - HPET registers a page that is mmap(2)-ed readonly by libc into + HPET registers a page that is mmap(2)-ed readonly by libc into userland application programs' address space as needed. - Measurements demonstrated modest improvements in gettimeofday(2) - performance, but not unexpectedly even the syscall-less HPET + Measurements demonstrated modest improvements in gettimeofday(2) + performance, but, not unexpectedly, even the syscall-less HPET timecounter is slower than invoking a syscall for RDTSC.

Some not strictly interwined but related code is the time-bound sleep implementation. Handling of races between callouts and the top-half code that sets and processes the timeouts depended on the many fine details of the - callout_stop(9) KPI (kernel programming interface). In + callout_stop(9) KPI (kernel programming interface). In particular, races or unpunctual KPI changes usually result in the "catch-all" unkillable thread state with the "-" waitchain bug. The sleepqueue timeout code was @@ -1440,10 +1440,10 @@

UEFI (Unified Extensible Firmware Interface) specifies two kinds of services for use by operating systems. Boot Services are - designed for OS loaders to be able to load and initialize kernels, - while Runtime Services are supposed to be used by the kernels + designed for OS loaders to load and initialize kernels, + while Runtime Services are meant to be used by kernels during regular system operations. The boot and runtime phases are - explicitely separated. During boot, when loaders are executed, + explicitly separated. During boot, when loaders are executed, the machine configuration is owned by UEFI. During runtime, the kernel manages the configuration, but needs to inform the firmware about any changes that are made.

@@ -1453,7 +1453,7 @@ &os; codebase. For instance, the firmware notification of the future runtime configuration must be done while the loader is effectively still in control. In technical terms, the - SetVirtualAddressMap() call must be made with the 1:1 + SetVirtualAddressMap() call must be made with the 1:1 physical:virtual mapping on amd64 systems, which for &os; means that the call can be only issued by the loader. But the loader needs to know intimate details of the kernel address map to @@ -1474,31 +1474,31 @@ centered around utilizing the direct address map (DMAP) on amd64, which currently always exists and linearly maps at least the lower 4G of physical addresses at some KVA location. - It was supposed that kernel would export the DMAP details like + It was supposed that the kernel would export the DMAP details like linear base and guaranteed size for loader from its ELF image, and provide the needed overflow map if the DMAP cannot completely serve. Unfortunately, two show-stopper bugs were discovered with this approach.

First, EDK-based firmwares apparently require that the +

First, EDK-based firmware apparently requires that the runtime mapping exists simultaneously with the physical - mapping for the SetVirtualAddressMap() call. Second, there + mapping for the SetVirtualAddressMap() call. Second, there were references from other open-source projects mentioning - that some firmwares required the presence of the physical - mapping during runtime call. Effectively, this forces both + that some firmware required the presence of the physical + mapping during the runtime call. Effectively, this forces both kernel and loader to provide both mappings for all runtime calls.

With such restrictions, informing the firmware about the details of the kernel address space only adds useless work. We could just as easily establish the 1:1 physical mapping - during runtime and get rid of SetVirtualAddressMap() entirely. + during runtime and get rid of SetVirtualAddressMap() entirely. This approach was coded and the kernel interface to access runtime services is based on it.

During the development, in particular, when trying to make +

During the development, particularly when trying to make the loader modifications, it was quickly realized that there - were no fault-reporting facilities in loader.efi. Machine + were no fault-reporting facilities in loader.efi. Machine exceptions resulted in a silent hang. Curiosly, in such a situation the Intel firmware outputs the error code over the serial port on 115200/8/1 settings, regardless of UEFI console @@ -1506,11 +1506,11 @@ Unfortunately, the error code alone is not enough to diagnose most problems.

A primitive fault reporter was written for loader.efi on +

A primitive fault reporter was written for loader.efi on amd64, which intercepts exceptions from the firmware IDT and - dumps the machine state to the loader console. Due to + dumps the machine state to the loader console. Due to the complexity of the interception and possible bugs which might - do more harm than good there, the dumper is only activated by + do more harm than good there, the dumper is only activated with explicit administrator action.

Note that the described work only provides the kernel @@ -1546,9 +1546,9 @@ respective branches, among other things.

The &os; Release Engineering Team continued the 11.0-RELEASE - cycle which was planned to be released in September, but as - result of several last-minute issues that needed to be - addressed, the final release announcement was delayed.

+ cycle which was planned to be released in September, but as a + result of several last-minute issues, + the final release announcement was delayed.

@@ -1595,31 +1595,31 @@

We are sad to report that both GSoc projects failed. The - libdevq project was abandonned as the student + libdevq project was abandoned as the student disappeared. The kernel project was incomplete because the student could not work for personal reasons. He plans to resume work and complete the task, even though GSoC 2016 is finished.

The X.org server version 1.18.4 together with updates for +

X.org server version 1.18.4 and updates for the xf86-video-ati and xf86-video-intel DDX - drivers are ready for wider testing and a CFT will be sent out - shortly. These updates are required in order to use newer DRM + drivers are ready for wider testing. A CFT will be sent out + shortly. These updates are required to use newer DRM versions.

The missing functionality from libdrm that is needed by the amdgpu driver has been added. These - chances will be committed to the ports tree shortly after the + changes will be committed to the ports tree shortly after the xorg-server update.

DRM from Linux 4.8 was ported to the drm-next branch. This branch should be used for radeon and - amdgpu cards; for i915 cards the - drm-next-4.7 should be used, due to instabilities in - the intel driver in drm-next branch.

+ amdgpu cards. The drm-next-4.7 branch should be used + for i915 cards due to instabilities in + the intel driver in drm-next branch.

Johannes Lundberg has been working on getting the Wayland - enviroment up and running on &os;. The Wayland ports are in + environment running on &os;. The Wayland ports are in a working state except for the Weston compositor.

The current Weston port (from DragonFlyBSD) might be @@ -1675,7 +1675,7 @@

Our work is 100% funded by your donations. Our spending budget for 2016 is $1,250,000 and we have raised $271,500 so far. Our Q1-Q3 financial reports will be posted in early - November. As you can see, we need your donations in order to + November. As you can see, we need your donations to continue supporting &os; at our current level. Please consider making a donation at https://www.FreeBSDfoundation.org/donate/.

@@ -1685,22 +1685,22 @@

The Foundation improves &os; by funding software development projects approved through our proposal submission process, and our internal software developer staff members. Two - Foundation-funded projects continued last quarter; one project + Foundation-funded projects continued last quarter: one project is to port NetBSD's blacklistd daemon and related elements to &os;, and the second is phase two of the &os;/arm64 port.

Foundation staff members were responsible for many changes - over the quarter. Kostik Belousov accomplished the following + over the quarter. Kostik Belousov accomplished this work last quarter:

Provided kernel support for EFI Runtime Services calls.
Implemented getttimeofday(2) purely in userspace for HPET +
Implemented getttimeofday(2) purely in userspace for HPET timers
Implemented fdatasync(2)
Implemented fdatasync(2)
Improved the locking of the time keeping code
Improved the process management and ptrace code

Ed Maste, our Project Development Director, accomplished the - following work last quarter:

Ed Maste, our Project Development Director, accomplished this + work last quarter:

Worked on &os;/arm64 issues and Cavium ThunderX @@ -1725,7 +1725,7 @@
Switched to using LLVM's libunwind in the base system
Improved the reproducibility of builds in &os; base system +
Improved the reproducibility of builds in the &os; base system and ports
Reviewed the blacklistd work that is in @@ -1794,9 +1794,9 @@
A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others - using &os;; producing advocacy literature to teach people about - &os; and easing the path to starting out with &os; and contributing - to the Project; and attending and getting other &os; + using &os;, producing advocacy literature to teach people about + &os; and ease the path to starting out with &os;, contributing + to the Project, and attending and getting other &os; contributors to volunteer to run &os; events, staff &os; tables, and give &os; presentations.
@@ -1910,7 +1910,7 @@
Other Stuff We Did
-
We welcomed Kylie Liange and Philip Paeps to the Board of +
We welcomed Kylie Liang and Philip Paeps to the Board of Directors. More information and interviews can be found at: https://www.FreeBSDfoundation.org/blog/FreeBSD-foundation-welcomes-new-board-members/.
@@ -2119,13 +2119,13 @@
The upstream lld project has now implemented nearly all of the functionality required to link the amd64 - FreeBSD base system, including the kernel. The boot loader + &os; base system, including the kernel. The boot loader components and rescue utilities currently do not build with lld.
- Merge lld to FreeBSD head as part of the Clang + Merge lld to &os; head as part of the Clang 3.9.0 import. Request a ports exp-run with lld installed as