From owner-svn-doc-head@freebsd.org Sat Aug 24 18:56:16 2019 Return-Path: Delivered-To: svn-doc-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B1EBBC7467; Sat, 24 Aug 2019 18:56:16 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46G6tS52rcz40yj; Sat, 24 Aug 2019 18:56:16 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8E6C1979C; Sat, 24 Aug 2019 18:56:16 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x7OIuGJw049046; Sat, 24 Aug 2019 18:56:16 GMT (envelope-from trasz@FreeBSD.org) Received: (from trasz@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x7OIuG03049044; Sat, 24 Aug 2019 18:56:16 GMT (envelope-from trasz@FreeBSD.org) Message-Id: <201908241856.x7OIuG03049044@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: trasz set sender to trasz@FreeBSD.org using -f From: Edward Tomasz Napierala Date: Sat, 24 Aug 2019 18:56:16 +0000 (UTC) To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r53319 - head/en_US.ISO8859-1/htdocs/news/status X-SVN-Group: doc-head X-SVN-Commit-Author: trasz X-SVN-Commit-Paths: head/en_US.ISO8859-1/htdocs/news/status X-SVN-Commit-Revision: 53319 X-SVN-Commit-Repository: doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-doc-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the doc tree for head List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Aug 2019 18:56:16 -0000 Author: trasz Date: Sat Aug 24 18:56:16 2019 New Revision: 53319 URL: https://svnweb.freebsd.org/changeset/doc/53319 Log: Add Quarterly Status Report for 2019Q2. Reviewed by: bcr (earlier version) Differential Revision: https://reviews.freebsd.org/D21381 Added: head/en_US.ISO8859-1/htdocs/news/status/report-2019-04-2019-06.xml (contents, props changed) Modified: head/en_US.ISO8859-1/htdocs/news/status/Makefile Modified: head/en_US.ISO8859-1/htdocs/news/status/Makefile ============================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/Makefile Sat Aug 24 17:03:34 2019 (r53318) +++ head/en_US.ISO8859-1/htdocs/news/status/Makefile Sat Aug 24 18:56:16 2019 (r53319) @@ -83,6 +83,7 @@ XMLDOCS+= report-2017-10-2017-12 XMLDOCS+= report-2018-01-2018-09 XMLDOCS+= report-2018-09-2018-12 XMLDOCS+= report-2019-01-2019-03 +XMLDOCS+= report-2019-04-2019-06 XSLT.DEFAULT= report.xsl Added: head/en_US.ISO8859-1/htdocs/news/status/report-2019-04-2019-06.xml ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2019-04-2019-06.xml Sat Aug 24 18:56:16 2019 (r53319) @@ -0,0 +1,2458 @@ + + + + + + + + + + April-June + + 2019 + + +
+ Introduction + +

This quarter our report includes + some interesting topics easily accessible to anyone, even if + you are not a programmer: we report the link to a presentation + of the 2019 FreeBSD survey results at BSDCan 2019 and describe + an interesting experience of a 3-person hackaton, which might + encourage you to host one yourself, possibly with more participants. + We also provide some up to date information about the status + of our IRC channels.

+ +

For those who have some more technical skills, we give some + news about the role of git in the FreeBSD project, describe + the status of some tools to hunt bugs or enhance security and + announce a clone of sysctl.

+ +

Finally, those who are more experienced with programming will + probably be interested in the great work that has been done + with drivers: in particular, an aknowledgement is due to Alan + Somers for having started to bring up to date our FUSE + implementation, which was about 11 years behind. Other important + improvements include a more user-friendly experience with + trackpoints and touchpads enabled by default, much low level + work on graphics, many new bhyve features, updates to the + linux compatibility layer, various kernel improvements.

+ +

Have a nice read!
+ + -- Lorenzo Salvadore

+
+ + + team + + &os; Team Reports + +

Entries from the various official and semi-official teams, + as found in the Administration + Page.

+ + + + proj + + Projects + +

Projects that span multiple categories, from the kernel and userspace + to the Ports Collection or external projects.

+ + + + kern + + Kernel + +

Updates to kernel subsystems/features, driver support, + filesystems, and more.

+ + + + arch + + Architectures + +

Updating platform-specific features and bringing in support + for new hardware platforms.

. + + + + bin + + Userland Programs + +

Changes affecting the base system and programs in it.

+ + + + ports + + Ports + +

Changes affecting the Ports Collection, whether sweeping + changes that touch most of the tree, or individual ports + themselves.

+ + + + doc + + Documentation + +

Noteworthy changes in the documentation tree or new external + books/documents.

+ + + + misc + + Miscellaneous + +

Objects that defy categorization.

+ + + + third + + Third-Party Projects + +

Many projects build upon &os; or incorporate components of + &os; into their project. As these projects may be of interest + to the broader &os; community, we sometimes include brief + updates submitted by these projects in our quarterly report. + The &os; project makes no representation as to the accuracy or + veracity of any claims in these submissions.

+ + + + Release Engineering Team + + + + FreeBSD Release Engineering Team + re@FreeBSD.org + + + + + FreeBSD 11.3-RELEASE schedule + FreeBSD 11.3-RELEASE announcement + FreeBSD 12.1-RELEASE schedule + FreeBSD development snapshots + + + +

The FreeBSD Release Engineering Team is responsible for + setting + and publishing release schedules for official project + releases + of FreeBSD, announcing code freezes and maintaining the + respective branches, among other things.

+ +

During the second quarter of 2019, the FreeBSD Release + Engineering team + started the 11.3-RELEASE cycle, with the code slush + starting May 3rd. + Throughout the cycle, there were three BETA builds and + three RC builds, + all of which in line with the originally-published + schedule. The final RC + build started June 28th, with the final release build + targeted for July 5th.

+ +

FreeBSD 11.3-RELEASE will be the fourth release from the + stable/11 + branch, building on the stability and reliability of + 11.2-RELEASE.

+ +

The FreeBSD Release Engineering Team also published the + schedule for the + 12.1-RELEASE, targeted to start September 6th. One + important thing to note + regarding the published schedule is it excludes a hard + freeze on the + stable/12 branch, as a test run for eliminating + code freezes entirely during + a release cycle. Commits to what will be the + releng/12.1 branch will still + require explicit approval from the Release Engineering + Team, however.

+ +

Additionally throughout the quarter, several development + snapshots builds + were released for the head, stable/12, + and stable/11 branches.

+ +

Much of this work was sponsored by the FreeBSD Foundation + and Rubicon + Communications, LLC (Netgate).

+ + + + + + + Ports Collection + + + + René Ladan + portmgr-secretary@FreeBSD.org + + + FreeBSD Ports Management Team + portmgr@FreeBSD.org + + + + + About FreeBSD Ports + Contributing to Ports + FreeBSD Ports Monitoring + Ports Management Team">Ports Management Team + + + +

The following was done during the last quarter by portmgr + to keep things in + the Ports Tree going:

+ +

During the last quarter the number of ports rose to just + under 37,000. At the + end of the quarter, there were 2146 open PRs and 7837 + commits (excluding 499 on + the quarterly branch) from 172 committers. This shows a + slight decrease in + activity compared to previous quarter.

+ +

People come and go, last quarter we welcomed Pedro Giffuni + (pfg@), Piotr Kubaj + (pkubaj@) and Hans Petter Selasky (hselasky@). Pedro and + Hans Petter were + already active as src committers. We said goodbye to + gordon@, kan@, tobez@, + and wosch@.

+ +

On the infrastructure side, a new USES=cabal was + introduced and various default + versions were updated: MySQL to 5.7, Python to 3.6, Ruby + to 2.5, Samba to 4.8 + and Julia gained a default version of 1.0. The web + browsers were also updated: + Firefox to 68.0 and Chromium to 75.0.3770.100

+ +

During the last quarter, antoine@ ran a total of 41 + exp-runs to test various + package updates, bump the stack protector level to + "strong", switch the default + Python version to 3.6 as opposed to 2.7, remove sys/dir.h + from base which has + been deprecated for over 20 years, and convert all Go + ports to USES=go.

+ + + + + + + FreeBSD Core Team + + + + FreeBSD Core Team + core@FreeBSD.org + + + + +

The FreeBSD Core Team is the governing body of FreeBSD.

+ +
    +
  • Core approved source commit bits for Doug Moore (dougm), + Chuck Silvers + (chs), Brandon Bergren (bdragon), and a vendor commit bit + for Scott + Phillips (scottph).
    • +
+ +
    +
  • The annual developer survey closed on 2019-04-02. Of the + 397 + developers, 243 took the survey with an average completion + time of 12 + minutes. The public survey closed on 2019-05-13. It was + taken by + 3637 users and had a 79% completion rate. + A + presentation of the survey results + took place at BSDCan 2019.
    • +
+ +
    +
  • The core team voted to appoint a working group to explore + transitioning our source code 'source of truth' from + Subversion to + Git. Core asked Ed Maste to chair the group as Ed has been + researching this topic for some time. For example, Ed gave + a + MeetBSD 2018 talk on the topic.
    • +
+ +

+ There is a variety of viewpoints within core regarding + where and how + to host a Git repository, however core feels that Git is + the prudent + path forward.

+ +
    +
  • The project received many Season of Docs submissions and + picked a top + candidate. Google will announce the accepted technical + writer + projects on 2019-08-06. We are hoping for lots of new and + refreshed + man pages.
    • +
+ + + + + + + Continuous Integration + + + + Jenkins Admin + jenkins-admin@FreeBSD.org + + + Li-Wen Hsu + lwhsu@FreeBSD.org + + + + + FreeBSD Jenkins Instance + FreeBSD CI artifact archive + FreeBSD Jenkins wiki + freebsd-testing Mailing List + freebsd-ci Repository + Tickets related to freebsd-testing@ + Hosted CI wiki + FreeBSD CI weekly report + + + +

The FreeBSD CI team maintains continuous integration + system and related tasks + for the FreeBSD project. The CI system regularly checks + the committed changes + can be successfully built, then performs various tests and + analysis of the + results. The results from build jobs are archived in an + artifact server, for + the further testing and debugging needs. The CI team + members examine the + failing builds and unstable tests, and work with the + experts in that area to + fix the code or adjust test infrastructure. The details + are of these efforts + are available in the weekly CI reports.

+ +

The + FCP + for CI policy + is in "feedback" state, please provide any comments to + freebsd-testing@ or + other suitable lists.

+ +

We had a testing working group in 201905 + DevSummit

+ +

Please see freebsd-testing@ related tickets for more + information.

+ +

Work in progress:

+ +
    +
  • Fixing the failing test cases and builds
    • + +
  • Adding drm ports building test against -CURRENT
    • + +
  • Adding powerpc64 tests job: https://github.com/freebsd/freebsd-ci/pull/33
    • + +
  • Implementing automatic tests on bare metal hardware
    • + +
  • Extending and publishing the embedded testbed
    • + +
  • Planning for running ztest and network stack tests
    • + +
  • Help more 3rd software get CI on FreeBSD through a hosted + CI solution
    • +
+ + + + + + + FreeBSD Graphics Team status report + + + + FreeBSD Graphics Team + x11@freebsd.org + + + Niclas Zeising + zeising@freebsd.org + + + + + Project GitHub page + + + +

The FreeBSD X11/Graphics team maintains the lower levels + of the FreeBSD graphics + stack. + This includes graphics drivers, graphics libraries such as + the + MESA OpenGL implementation, the X.org xserver with related + libraries and + applications, and Wayland with related libraries and + applications.

+ +

In the last report, half a year ago, several updates and + changes had been made + to the FreeBSD graphics stack.

+ +

To further improve the user experience, and to improve + input device handling, + evdev was enabled in the default configuration in late + 2018. Building on that, + we have enabled IBM/Lenovo trackpoints and elantech and + synaptics touchpads by + default as well.

+ +

The input device library libinput has been updated as the + last in a series of + updates bringing the userland input stack up to date. + This is work that was started in 2018.

+ +

We have made several improvements to the drm kernel + drivers. + A long-standing memory leak in the Intel (i915) driver has + been fixed, and + several other updates and improvements have been made to + the various drm + kernel driver components.

+ +

A port of the drm kernel drivers using the 5.0 Linux + kernel sources has been + created and committed to FreeBSD ports as + graphics/drm-devel-kmod. + This driver requires a recent Linux KPI and is only + available on recent + versions of FreeBSD CURRENT.

+ +

This version of the driver contains several development + improvements. + The generic drm (drm.ko) driver as well as the i915 + (i915kms.ko) driver + can now be unloaded and reloaded to ease in development + and testing. + This causes issues with the virtual consoles, however, so + an SSH connection is + recommended. + To aid debugging i915kms.ko use of debugfs has + been improved, but there are + still limitations preventing it from being fully + functional. + Since debugfs is based on pseudofs it is possible that + this will prevent a fully + functional debugfs in its current state, so we might have + to look into adding + the required functionality to pseudofs or use another + framework.

+ +

The new in-kernel drm driver for VirtualBox, + vboxvideo.ko has been ported from + Linux. + Support is currently an experimental work in progress. + For example the virtual console won't update after loading + the driver, but X- + and Wayland-based compositors are working.

+ +

Mesa has been updated to 18.3.2 and switched from using + devel/llvm60 to use + the Ports default version of llvm, currently + devel/llvm80.

+ +

Several userland Xorg drivers, applications, and libraries + have been updated, + and other improvements to the various userland components + that make up the + Graphics Stack have been made.

+ +

We have also continued our regularly scheduled bi-weekly + meetings, although work + remains in sending out timely meeting minutes afterwards.

+ +

People who are interested in helping out can find us on + the x11@FreeBSD.org + mailing list, or on our gitter chat: https://gitter.im/FreeBSDDesktop/Lobby. + We are also available in #freebsd-xorg on EFNet.

+ +

We also have a team area on GitHub where our work + repositories can be found: + https://github.com/FreeBSDDesktop

+ + + + + + + IRC Admin + + + + IRC Admin + irc@FreeBSD.org + + + + +

The FreeBSD IRC Admin team manages the FreeBSD Project's + presence + and activity on the freenode IRC network, looking after:

+ +
    +
  • Registration and management of channels within the + official namespace (#freebsd*)
    • + +
  • Channel moderation
    • + +
  • Liaising with freenode staff
    • + +
  • Allocating freebsd/* hostmask cloaks for users
    • + +
  • General user support relating to channel management
    • +
+ +

+ While the FreeBSD Project does not _currently_ endorse IRC + as an + official support channel [1][2], as it has not been able + to guarantee + a consistent or positive user experience, IRC Admin has + been working + toward creating a high quality experience, by + standardising channel + administration and moderation expectations, and ensuring + the projects + ability to manage all channels within its namespace.

+ +

In the last quarter, IRC Admin:

+ +
    +
  • Cleaned up (deregistered) registrations for channels that + were defunct, + stale, out of date, or had founders that were inactive + (not seen for > 1 + year). Channels that were found to be otherwise active + have been retained. + FreeBSD now has ~40 channels registered from a previous + total of over 150.
    • + +
  • Documented baseline configuration settings in the Wiki for + channels, + including ChanServ settings, channel modes, registration + policy, etc.
    • + +
  • Established multiple documented methods for reporting user + abuse + or other channel issues to IRC Admin for resolution
    • +
+ +

+ Upcoming changes:

+ +
    +
  • Work with existing #freebsd* channels to + standardise channel management, + settings and access.
    • + +
  • Migrate, forward and/or consolidate existing or duplicate + #freebsd* + channels to channels with a standard naming convention.
    • + +
  • Work with unofficial ##freebsd* channels to + migrate them to the official + #freebsd* channels if suitable
    • + +
  • Update existing IRC-related website and documentation + sources the describe + the official state of project managed IRC presence on + freenode.
    • +
+ +

+ Lastly, and to repeat a previous call, while the vast + majority of + the broader user community interacts on the freenode IRC + network, + the FreeBSD developer presence still needs to be + significantly + improved on freenode.

+ +

There are many opportunities to be had by increasing the + amount and + quality of interaction between FreeBSD users and + developers, both + in terms of developers keeping their finger on the pulse + of the + community and in encouraging and cultivating greater + contributions + to the Project over the long term.

+ +

It is critical to have a strong developer presence amongst + users, + and IRC Admin would like again to call on all developers + to join + the FreeBSD freenode channels to increase that presence.

+ +

Users are invited to /join #freebsd-irc on the + freenode IRC network + if they have questions, ideas, constructive criticism, and + feedback + on how the FreeBSD Project can improve the service and + experience + it provides to the community on IRC.

+ +

[1] https://www.freebsd.org/community/irc.html + [2] + https://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/support.html#irc

+ + + + + + + bhyve - Live Migration + + + + Elena Mihailescu + elenamihailescu22@gmail.com + + + Darius Mihai + dariusmihaim@gmail.com + + + Mihai Carabas + mihai@freebsd.org + + + + + Github wiki - How to Live and Warm Migrate a bhyve guest + Github - Warm Migration branch + Github - Live Migration branch + + + +

The Migration feature uses the Save/Restore feature to + migrate a bhyve guest + from a FreeBSD host to another FreeBSD host. To migrate a + bhyve guest, + one needs to start an empty guest on the destination host + from a shared guest + image using the bhyve tool with the -R option + followed by the source host + IP and the port to listen to migration request. On the + source host, the + migration is started by executing the bhyvectl command + with the --migrate + or --migrate-live option, followed by the + destination host IP and the + port to send to the messages.

+ +

New features added:

+ +
    +
  • Clear the dirty bit after each migration round
    • + +
  • Extend live migration to highmem segment
    • +
+ +

+ Future tasks:

+ +
    +
  • Refactor live migration branch
    • + +
  • Rebase live migration
    • + +
  • Extend live migration to unwired memory
    • +
+ + + + + Matthew Grooms + + + + + + bhyve - Save/Restore + + + + Elena Mihailescu + elenamihailescu22@gmail.com + + + Darius Mihai + dariusmihaim@gmail.com + + + Mihai Carabas + mihai@freebsd.org + + + + + Github repository for the snapshot feature for bhyve + Github wiki - How to Save and Restore a bhyve guest + Github wiki - Suspend/resume test matrix + Phabricator review - bhyve Snapshot Save and Restore + + + +

The Save/Restore for bhyve feature is a suspend and resume + facility added to the + FreeBSD/amd64's hypervisor, bhyve. The bhyvectl tool is + used to save the guest + state in three files (a file for the guest memory, a file + for the states of + various devices and the state of the CPU, and another one + for some metadata that + is used in the restore process). + To suspend a bhyve guest, the bhyvectl tool must be run + with the --suspend + <state_file_name> + option followed by the guest name.

+ +

To restore a bhyve guest from a checkpoint, one simply has + to add the -r option + followed by the main state file (the same file that was + given to the --suspend + option for bhyvectl) when starting the VM.

+ +

New features added:

+ +
    +
  • Open ticket on Phabricator
    • + +
  • Apply feedback received from community
    • +
+ +

+ Future tasks:

+ +
    +
  • Add suspend/resume support for nvme
    • + +
  • Add suspend/resume support for virtio-console
    • + +
  • Add suspend/resume support for virtio-scsi
    • + +
  • Add TSC offsetting for restore for AMD CPUs
    • +
+ + + + + Matthew Grooms + + + + + + ENA FreeBSD Driver Update + + + + Michal Krawczyk + mk@semihalf.com + + + Maciej Bielski + mba@semihalf.com + + + Marcin Wojtas + mw@semihalf.com + + + + + ENA README + + + +

ENA (Elastic Network Adapter) is the smart NIC available + in the + virtualized environment of Amazon Web Services (AWS). The + ENA + driver supports multiple transmit and receive queues and + can handle + up to 100 Gb/s of network traffic, depending on the + instance type + on which it is used.

+ +

ENAv2 has been under development for FreeBSD, similar to + Linux + and DPDK. Since the last update internal review and + improvements + of the patches were done, followed by validation on + various AWS + instances.

+ +

Completed since the last update:

+ +
    +
  • Upstream of the ENAv2 patches - revisions + r348383 + - + r348416 + introduce a major driver upgrade to version v2.0.0. Along + with various fixes + and improvements, the most significant features are LLQ + (Low Latency Queues) + and independent queues reconfiguration using sysctl + commands.
    • + +
  • Implement NETMAP support for ENA
    • +
+ +

+ Todo:

+ +
    +
  • Internal review and upstream of NETMAP support
    • +
+ + + + + Amazon.com Inc + + + + + + FUSE + + + + Alan Somers + asomers@FreeBSD.org + + + + +

FUSE (File system in USErspace) allows a userspace program + to + implement a file system. It is widely used to support + out-of-tree file + systems like NTFS, as well as for exotic pseudo file + systems like + sshfs. FreeBSD's fuse driver was added as a GSoC project + in 2012. + Since that time, it has been largely neglected. The FUSE + software is + buggy + and out-of-date. Our implementation is about 11 years + behind.

+ +

During Q2 I nearly finished the FUSE overhaul that I + begain in Q1. I raised + the protocol level from 7.8 to 7.23, fixed many bugs (see + 199934, + 216391, + 233783, + 234581, + 235773, + 235774, + 235775, + 236226, + 236231, + 236236, + 239291, + 236329, + 236379, + 236381, + 236405, + 236327, + 236466, + 236472, + 236473, + 236474, + 236530, + 236557, + 236560, + 236647, + 236844, + 237052, + 237181, + 237588, + and + 238565), + and added + the following features:

+ +
    +
  • Optional kernel-side permissions checks (`-o + default_permissions`)
    • + +
  • Implement VOP_MKNOD, VOP_BMAP, and + VOP_ADVLOCK
    • + +
  • Allow interrupting FUSE operations
    • + +
  • Support named pipes and unix-domain sockets in fusefs file + systems
    • + *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***