Date: Mon, 15 Nov 2021 17:47:44 +0100 From: Daniel Ebdrup Jensen <debdrup@FreeBSD.org> To: hackers@FreeBSD.org Cc: current@FreeBSD.org, stable@FreeBSD.org Subject: FreeBSD Quarterly Status Report - Third Quarter 2021 Message-ID: <20211115164744.chsesrxkz3bb4wct@nerd-thinkpad.local>
next in thread | raw e-mail | index | archive | help
--66dsezfsnajsewcm
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
FreeBSD Quarterly Status Report 3rd Quarter 2021
This report covers FreeBSD related projects for the period between July and
September, and is the third of four planned reports for 2021, and contains =
42
entries.
The third quarter of 2021 was quite active in lots of different areas, so t=
he
report covers a bunch of interesting work including but not limited to boot
performance, compile-time analysis, hole-punching support, various drivers,=
ZFS
raidz expansion, an update to the sound mixer, and many more.
Regrettably, the status report got a bit delayed, but we hope that the apho=
rism
better late than never can apply here.
Yours,
Daniel Ebdrup Jensen, with status hat on.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Table of Contents
=E2=80=A2 FreeBSD Team Reports
=E2=96=A1 FreeBSD Foundation
=E2=96=A1 FreeBSD Release Engineering Team
=E2=96=A1 Cluster Administration Team
=E2=96=A1 Continuous Integration
=E2=96=A1 Ports Collection
=E2=96=A1 Documentation Engineering Team
=E2=96=A1 FreeBSD Website Revamp - WebApps working group
=E2=80=A2 Projects
=E2=96=A1 Boot Performance Improvements
=E2=96=A1 Git Migration Working Group
=E2=96=A1 LLDB Debugger Improvements
=E2=96=A1 Linux compatibility layer update
=E2=96=A1 Sound mixer improvements
=E2=96=A1 Base System OpenSSH Update
=E2=80=A2 Kernel
=E2=96=A1 Arm64 improvements
=E2=96=A1 FreeBSD on Microsoft HyperV and Azure
=E2=96=A1 Improved amd64 UEFI boot
=E2=96=A1 ENA FreeBSD Driver Update
=E2=96=A1 Hole-punching support on FreeBSD
=E2=96=A1 Intel Networking on FreeBSD
=E2=96=A1 Intel Wireless driver support
=E2=96=A1 Microchip LAN743x mgb(4) Device Driver
=E2=96=A1 Fixes for msdosfs_rename VOP
=E2=96=A1 OpenZFS RAIDZ Expansion update
=E2=96=A1 RFC1191 support in IPSEC tunnels
=E2=96=A1 Realtek rtw88 support
=E2=96=A1 Marvell SDHCI improvements and ACPI support
=E2=96=A1 Stack gap handling improvements
=E2=96=A1 syzkaller on FreeBSD
=E2=96=A1 Using OCF in WireGuard
=E2=96=A1 Intel Volume Management Device driver update
=E2=80=A2 Ports
=E2=96=A1 Improve CPE Data in the Ports Collection
=E2=96=A1 Why do we need it?
=E2=96=A1 How can I help?
=E2=96=A1 Open Tasks
=E2=96=A1 FreeBSD Erlang Ecosystem Ports update
=E2=96=A1 KDE on FreeBSD
=E2=96=A1 OpenSearch on FreeBSD
=E2=96=A1 Valgrind - Official Support for FreeBSD has Landed
=E2=96=A1 Wine on FreeBSD
=E2=96=A1 Ztop
=E2=80=A2 Miscellaneous
=E2=96=A1 -CURRENT compilation time analysis
=E2=80=A2 Third-Party Projects
=E2=96=A1 Gitlab 14.3 available
=E2=96=A1 helloSystem
=E2=96=A1 Containers & FreeBSD: Pot, Potluck & Potman
=E2=96=A1 WireGuard on FreeBSD Stabilizes, Eyes Upstreaming
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
FreeBSD Team Reports
Entries from the various official and semi-official teams, as found in the
Administration Page.
FreeBSD Foundation
Links:
FreeBSD Foundation URL: https://www.FreeBSDfoundation.org
Technology Roadmap URL: https://freebsdfoundation.org/blog/technology-roadm=
ap/
Donate URL: https://www.FreeBSDfoundation.org/donate/
Foundation Partnership Program URL: https://www.FreeBSDfoundation.org/
FreeBSD-foundation-partnership-program
FreeBSD Journal URL: https://www.FreeBSDfoundation.org/journal/
Foundation News and Events URL: https://www.FreeBSDfoundation.org/
news-and-events/
Contact: Deb Goodkin <deb@FreeBSDFoundation.org>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to
supporting and promoting the FreeBSD Project and community worldwide. Donat=
ions
=66rom individuals and corporations are used to fund and manage software
development projects, conferences, and developer summits. We also provide
travel grants to FreeBSD contributors, purchase and support hardware to imp=
rove
and maintain FreeBSD infrastructure, and provide resources to improve secur=
ity,
quality assurance, and release engineering efforts. We publish marketing
material to promote, educate, and advocate for the FreeBSD Project, facilit=
ate
collaboration between commercial vendors and FreeBSD developers, and finall=
y,
represent the FreeBSD Project in executing contracts, license agreements, a=
nd
other legal arrangements that require a recognized legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
Fundraising Efforts
Fundraising last quarter wasn=E2=80=99t as spectacular as we were hoping. B=
ut, then
again, people tend to take vacations during the summer months, which makes =
it
that more difficult for our funding requests to go through the management c=
hain
for approvals.
So far this year we=E2=80=99ve raised $180,000 towards our $2,000,000 spend=
ing budget.
Why do we need so much money? Well, last year we decided to make more
significant software contributions to FreeBSD. In order to do that, we had =
to
grow our team. We developed a technology roadmap based on input we were
receiving from commercial users as well as market trends. Based on the road=
map,
we identified positions we needed to fill.
This year we=E2=80=99ve hired three full-time software developers, one full=
-time ARM
kernel developer, and one project manager. We also are funding wifi work
full-time and some other projects to help with FreeBSD on the desktop. You =
can
read about this effort to attract new users and contributors to the Project=
in
individual entries elsewhere in this status report.
Our growth wasn=E2=80=99t just in our technology team, but in our advocacy =
team too.
Here we hired a marketing coordinator and technical writer to provide more
educational and informational content. You=E2=80=99ll see in the Advocacy a=
nd Education
section below all the work we did to promote FreeBSD, provide community
engagement, education opportunities, and informative content to help pave t=
he
path to getting started with FreeBSD.
You=E2=80=99ll find out how we used your donations here and in individual e=
ntries
throughout this status report.
We=E2=80=99re passionate about supporting you, the FreeBSD community, but w=
e can=E2=80=99t do
it without your financial support.
Please consider making a donation to help us continue and increase our supp=
ort
for FreeBSD in 2021: https://www.FreeBSDfoundation.org/donate/.
We also have the Partnership Program, to provide more benefits for our larg=
er
commercial donors. Find out more information at https://
www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and share
with your companies!
OS Improvements
During the third quarter, Foundation staff and grant recipients committed 4=
20
src tree changes, 24 ports tree changes, and 11 doc tree changes. This
represents 38%, 48%, and 16% of src, port, and doc commits which identify a
sponsor.
You can read about Foundation-sponsored projects in individual quarterly re=
port
entries:
=E2=80=A2 Base System OpenSSH Update
=E2=80=A2 Fixes for msdosfs_rename VOP
=E2=80=A2 Hole Punching
=E2=80=A2 Improved amd64 UEFI boot
=E2=80=A2 Intel Wireless driver support
=E2=80=A2 LLDB Debugger Improvements
=E2=80=A2 Microchip LAN743x mgb(4) Device Driver
=E2=80=A2 OpenZFS RAIDZ Expansion update
=E2=80=A2 Using OCF in WireGuard
=E2=80=A2 syzkaller on FreeBSD
Foundation-sponsored arm64 work:
=E2=80=A2 Support for booting FreeBSD on the Arm Armv8-A AEM simulator
=E2=80=A2 sha256 instruction support to libmd(4) on arm64
=E2=80=A2 Initial work to support RAS, PAC and BTI on arm64
Continuous Integration and Quality Assurance
The Foundation provides a full-time staff member and funds projects to impr=
ove
continuous integration, automated testing, and overall quality assurance
efforts for the FreeBSD project.
See the separate Continuous Integration entry for details.
Supporting FreeBSD Infrastructure
The Foundation provides hardware and support for the Project. Last quarter,=
we
continued supporting the test cluster at Sentex, purchased a few needed dri=
ves
and spam filtering software for project email. We also set up a new and more
efficient hardware request/purchase process for clusteradm to use.
Partnerships and Commercial User Support
We met virtually with corporate users and started travelling again in late =
Q3
for some in-person meetings. The goals of the meetings are to facilitate
collaboration between commercial users and FreeBSD developers. We also met =
with
companies to discuss their needs and share that information with the Projec=
t.
FreeBSD Advocacy and Education
Much of our effort is dedicated to Project advocacy. This may involve
highlighting interesting FreeBSD work, producing literature, attending even=
ts,
or giving presentations. The goal of the literature we produce is to teach
people FreeBSD basics and help make their path to adoption or contribution
easier. Other than attending and presenting at events, we encourage and help
community members run their own FreeBSD events, give presentations, or staff
FreeBSD tables.
The FreeBSD Foundation sponsors many conferences, events, and summits around
the globe. These events can be BSD-related, open source, or technology even=
ts
geared towards underrepresented groups. We support the FreeBSD-focused even=
ts
to help provide a venue for sharing knowledge, working together on projects,
and facilitating collaboration between developers and commercial users. This
all helps provide a healthy ecosystem. We support the non-FreeBSD events to
promote and raise awareness of FreeBSD, to increase the use of FreeBSD in
different applications, and to recruit more contributors to the Project. We
finally made it back to our first in-person meeting with the Open Source Su=
mmit
in late September. We are also continuing to attend virtual events. In addi=
tion
to attending and planning virtual events, we are continually working on new
training initiatives and updating our selection of how-to guides to facilit=
ate
getting more folks to try out FreeBSD.
Check out some of the advocacy and education work we did last quarter:
=E2=80=A2 Participated as an Industry Partner for USENIX ATC and OSD, Jul=
y 14-16,
2021
=E2=80=A2 Participated as an Industry Partner for USENIX Security, August=
11-13, 2021
=E2=80=A2 Held a FreeBSD Friday: How to Track FreeBSD Using Git
=E2=80=A2 Sponsored and gave a talk at OpenFest 2020 in Sofia, Bulgaria, =
August
14-15, 2021
=E2=80=A2 Began planning for the November 2021 FreeBSD Vendor Summit to b=
e held
online, November 18-19
=E2=80=A2 Presented at APNIC on August 13-16, 2021
=E2=80=A2 Served as a Nonprofit In-Kind Sponsor of OSI=E2=80=99s POSI 202=
1, September 16,
2021
=E2=80=A2 Sponsored EuroBSDcon at the Silver Level. The event took place,=
September
17-19, 2021
=E2=80=A2 Presented at Open Source Summit, in Seattle, WA, September 27-3=
0, 2021
=E2=80=A2 Committed to be a Bronze Sponsor for the OpenZFS Summit
=E2=80=A2 New blog and video posts:
=E2=96=A1 Status of Online Conference Software on FreeBSD
=E2=96=A1 Meet the Summer 2021 Foundation Interns
=E2=96=A1 A Look at Profiling: FreeBSD Sort
=E2=96=A1 Meet the 2021 FreeBSD Google Summer of Code Students
=E2=96=A1 A Co-op Term at the FreeBSD Foundation
=E2=96=A1 Technology Roadmap
=E2=80=A2 Devstyler Interview with Deb Goodkin
=E2=80=A2 New Video How-to Guides on installing HelloSystem and installin=
g GhostBSD
=E2=80=A2 New Quick Start Guide on Printing on FreeBSD
=E2=80=A2 Committed to be a Media Sponsor for All Things Open
We help educate the world about FreeBSD by publishing the professionally
produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is
now a free publication. Find out more and access the latest issues at https=
://
www.FreeBSDfoundation.org/journal/.
You can find out more about events we attended and upcoming events at https=
://
www.FreeBSDfoundation.org/news-and-events/.
Legal/FreeBSD IP
The Foundation owns the FreeBSD trademarks, and it is our responsibility to
protect them. We also provide legal support for the core team to investigate
questions that arise.
Go to https://www.FreeBSDfoundation.org to find more about how we support
FreeBSD and how we can help you!
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
FreeBSD Release Engineering Team
Links:
FreeBSD 12.3-RELEASE schedule URL: https://www.freebsd.org/releases/12.3R/
schedule/
FreeBSD releases URL: https://download.freebsd.org/ftp/releases/ISO-IMAGES/
FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapsho=
ts/
ISO-IMAGES/
Contact: FreeBSD Release Engineering Team, <re@FreeBSD.org>
The FreeBSD Release Engineering Team is responsible for setting and publish=
ing
release schedules for official project releases of FreeBSD, announcing code
freezes and maintaining the respective branches, among other things.
Throughout the third quarter, several development snapshots builds were
released for the main, stable/13, and stable/12 branches. More work had been
done on various release-related tooling following the conversion from
Subversion to Git. Additionally, the team had drafted the proposed schedule=
for
the upcoming 12.3-RELEASE.
Sponsor: Rubicon Communications, LLC ("Netgate") Sponsor: The FreeBSD
Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Cluster Administration Team
Links:
Cluster Administration Team members URL: https://www.freebsd.org/administra=
tion
/#t-clusteradm
Contact: Cluster Administration Team <clusteradm@FreeBSD.org>
FreeBSD Cluster Administration Team members are responsible for managing the
machines the Project relies on to synchronise its distributed work and
communications. In this quarter, the team has worked on the following:
=E2=80=A2 Fixed www.FreeBSD.org mirror sync source
=E2=80=A2 Improvements to GeoDNS for {download,ftp,pkg}.FreeBSD.org
=E2=80=A2 Added IPv6 support
=E2=80=A2 More optimal mirror selection in Asia
=E2=80=A2 Finished installing a new mirror site in Japan, sponsored by Br=
oadband
Tower, Inc.
=E2=80=A2 Full mirror site (ftp, pkg, git, doc, dns,=E2=80=A6=E2=80=8B)
=E2=80=A2 The network and machine resources for this mirror are generousl=
y sponsored
by the Cloud and SDN Laboratory at BroadBand Tower, Inc., one of the
Internet data center service providers in Tokyo, Japan, with 300+ Gbps
international IP transit bandwidth
=E2=80=A2 Improved the retention script used for the artifact server of C=
I cluster to
offer a mix of the latest artifacts but also a selection of older artif=
acts
for comparison, space permitting
=E2=80=A2 Ongoing day to day cluster administration
=E2=80=A2 Replacing failed disks
=E2=80=A2 Babysitting pkgsync
Work in progress:
=E2=80=A2 Move pkg-master.nyi to new hardware
=E2=80=A2 Improve the package building infrastructure
=E2=80=A2 Review the service jails and service administrators operation
=E2=80=A2 Setup powerpc pkgbuilder/ref/universal machines
=E2=80=A2 Search for more providers that can fit the requirements for a g=
eneric
mirrored layout or a tiny mirror
=E2=80=A2 Upgrading public-facing cluster services from 12.2-STABLE to 13=
=2E0-STABLE
=E2=80=A2 Working with doceng@ to improve https://www.freebsd.org and htt=
ps://
docs.freebsd.org
=E2=80=A2 Improve the web service architecture
=E2=80=A2 Improve the cluster backup plan
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Continuous Integration
Links:
FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org
FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org
FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins
Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI
3rd Party Software CI URL: https://wiki.freebsd.org/3rdPartySoftwareCI
Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maau=
wg
FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci+ dev-ci
Mailing List URL: https://lists.freebsd.org/subscription/dev-ci
Contact: Jenkins Admin <jenkins-admin@FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
Contact: freebsd-testing Mailing List
Contact: IRC #freebsd-ci channel on EFNet
The FreeBSD CI team maintains the continuous integration system of the Free=
BSD
project. The CI system checks the committed changes can be successfully bui=
lt,
then performs various tests and analysis over the newly built results. The
artifacts from those builds are archived in the artifact server for further
testing and debugging needs. The CI team members examine the failing builds=
and
unstable tests and work with the experts in that area to fix the code or ad=
just
test infrastructure.
During the third quarter of 2021, we continued working with the contributors
and developers in the project to fulfil their testing needs and also keep
collaborating with external projects and companies to improve their products
and FreeBSD.
Important changes:
=E2=80=A2 The results of FreeBSD-main-amd64-build and FreeBSD-main-amd64-=
test jobs
are sent to the dev-ci mailing list
New jobs:
=E2=80=A2 Run tests with KASAN enabled for main on amd64
=E2=80=A2 Run tests with KMSAN enabled for main on amd64
Retired jobs:
=E2=80=A2 The jobs for stable/11 branch were removed after September 30th=
per FreeBSD
11.4 end-of-life
Work in progress and open tasks:
=E2=80=A2 Designing and implementing pre-commit CI building and testing (=
to support
the workflow working group)
=E2=80=A2 Designing and implementing use of CI cluster to build release a=
rtifacts as
release engineering does
=E2=80=A2 Collecting and sorting CI tasks and ideas here
=E2=80=A2 Testing and merging pull requests in the FreeBSD-ci repo
=E2=80=A2 Reducing the procedures of CI/test environment setting up for c=
ontributors
and developers
=E2=80=A2 Setting up the CI stage environment and putting the experimenta=
l jobs on it
=E2=80=A2 Setting up public network access for the VM guest running tests
=E2=80=A2 Implementing using bare metal hardware to run test suites
=E2=80=A2 Adding drm ports building tests against -CURRENT
=E2=80=A2 Planning to run ztest tests
=E2=80=A2 Adding more external toolchain related jobs
=E2=80=A2 Improving maturity of the hardware lab and adding more hardware=
under test
=E2=80=A2 Helping more software get FreeBSD support in its CI pipeline (W=
iki pages:
3rdPartySoftwareCI, HostedCI)
=E2=80=A2 Working with hosted CI providers to have better FreeBSD support
Please see freebsd-testing@ related tickets for more WIP information, and d=
on=E2=80=99t
hesitate to join the effort!
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Ports Collection
Links:
About FreeBSD Ports URL: https://www.FreeBSD.org/ports/
Contributing to Ports URL: https://docs.freebsd.org/en/articles/contributin=
g/
ports-contributing/
FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/
Ports Management Team URL: https://www.freebsd.org/portmgr/
Ports Tarball URL: http://ftp.freebsd.org/pub/FreeBSD/ports/ports/
Contact: Ren=C3=A9 Ladan <portmgr-secretary@FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>
The Ports Management Team is responsible for overseeing the overall directi=
on
of the Ports Tree, building packages, and personnel matters. Below is what
happened in the last quarter.
There are currently 46,500 ports in the Ports Tree according to FreshPorts.=
The
open PR count for ports is currently 2,588, of which 608 are unassigned. The
last quarter saw 9,833 commits on the "main" branch by 148 committers and 7=
43
commits on the quarterly branch by 54 committers. Compared to 2021q2, activ=
ity
mostly remained the same, the PR counts increased a bit but there were also
more commits to the quarterly branch.
During 2021q3, we welcomed Daniel Engberg (diizzy@) and Yasuhiro Kimura
(yasu@), and we said goodbye to culot@ and grog@ after their commit bits we=
re
taken in for safekeeping.
Two new uses were introduced: angr to help with testing Python-related ports
and mlt to help depending on mlt, a multimedia framework for TV broadcastin=
g.
Ruby saw minor updates for the 2.6, 2.7, and 3.0 series.
The "big" packages were also updated: pkg to 1.17.2, Firefox to 93.0, and
Chromium to 92.0.4515.159.
As usual, exp-runs were performed by antoine@ but only those of July were
recorded. There were 5 exp-runs to test adding CLOCK_\*_COARSE compatibility
definitions for Linux and to test ports updates.
Furthermore, rene@ gave a presentation "portmgr: behind the scenes" at
EuroBSDCon 2021 about how portmgr came to be and its daily activities.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Documentation Engineering Team
Links:
FreeBSD Documentation Project URL: https://www.freebsd.org/docproj/
FreeBSD Documentation Project Primer for New Contributors URL: link:https://
docs.freebsd.org/en/books/fdp-primer/
Documentation Engineering Team URL: https://www.freebsd.org/administration/#
t-doceng
Contact: FreeBSD Doceng Team <doceng@FreeBSD.org>
The doceng@ team is a body to handle some of the meta-project issues associ=
ated
with the FreeBSD Documentation Project; for more information, see FreeBSD
Doceng Team Charter.
During the last quarter, Philip Paeps (philip@) and Li-Wen Hsu (lwhsu@),
already src and ports committers, were granted documentation commit bits, b=
oth
now have all commit bit types. Ed Maste (emaste@) who is already a src
committer was granted a documentation commit bit. We also said goodbye to
Murray Stokely (murray@), G=C3=A1bor K=C3=B6vesd=C3=A1n (gabor@), Warren Bl=
ock (wblock@), and
Sevan Janiyan (sevan@). G=C3=A1bor K=C3=B6vesd=C3=A1n (gabor@) and Warren B=
lock (wblock@) are
now former members of the doceng@ team; we thank them for their years of
service.
Implicit (blanket) approvals were documented in the Committers Guide. That =
does
not cover all cases, but doceng@ encourages any FreeBSD committer from ports
and src to contribute to the doc tree.
All ports/packages misc/freebsd-doc-* were updated. They have the entire
documentation set from the FreeBSD Documentation Project, like Handbook, FA=
Q,
articles, and more.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
FreeBSD Website Revamp - WebApps working group
Contact: Sergio Carlavilla <carlavilla@FreeBSD.org>
Working group in charge of creating the new FreeBSD Documentation Portal and
redesigning the FreeBSD main website and its components. FreeBSD developers=
can
follow and join the working group on the FreeBSD Slack channel #wg-www21. T=
he
work will be divided into four phases:
1. Redesign of the Documentation Portal
Create a new design, responsive and with global search. (Work in progre=
ss,
almost complete)
2. Redesign of the Manual Pages on web
Scripts to generate the HTML pages using mandoc. (Work in progress)
3. Redesign of the Ports page on web
Ports scripts to create an applications portal. (Not started)
4. Redesign of the FreeBSD main website
New design, responsive and dark theme. (Not started)
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Projects
Projects that span multiple categories, from the kernel and userspace to the
Ports Collection or external projects.
Boot Performance Improvements
Links:
Wiki page URL: https://wiki.freebsd.org/BootTime
OS boot time comparison URL: https://www.daemonology.net/blog/
2021-08-12-EC2-boot-time-benchmarking.html
Contact: Colin Percival <cperciva@FreeBSD.org>
Colin Percival is coordinating an effort to speed up the FreeBSD boot proce=
ss.
For benchmarking purposes, he is using an EC2 c5.xlarge instance as a refer=
ence
platform and is measuring the time between when the virtual machine enters =
the
EC2 "running" state and when it is possible to SSH into the instance.
This work started in 2017, leading to a conference presentation, "Profiling=
the
FreeBSD kernel boot", and quickly yielded roughly 4850 ms of improvements
(starting from a baseline of about 30 seconds).
Since June, another roughly 9790 ms of time has been shaved off the boot
process, taking it down to approximately 15 seconds. There is still more wo=
rk
to be done; in particular, while the loader and kernel have been profiled, =
the
TSLOG system Colin is using does not currently support userland profiling.
Issues are listed on the wiki page as they are identified; the wiki page al=
so
has instructions for performing profiling. Users are encouraged to profile =
the
boot process on their own systems, in case they experience delays which don=
=E2=80=99t
show up on the system Colin is using for testing.
This work is supported by Colin=E2=80=99s FreeBSD/EC2 Patreon.
Sponsor: https://www.patreon.com/cperciva
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Git Migration Working Group
Links:
Git for FreeBSD development wiki URL: https://wiki.freebsd.org/Git
Git transition wiki URL: https://wiki.freebsd.org/GitTransition
doc git repo web URL: https://cgit.FreeBSD.org/doc
ports git repo web URL: https://cgit.FreeBSD.org/ports
src (base system) git repo web URL: https://cgit.FreeBSD.org/src
Committers guide Git primer URL: https://docs.freebsd.org/en/articles/
committers-guide/#git-primer
Handbook Using Git appendix URL: https://docs.freebsd.org/en/books/handbook/
mirrors/#git
Game of Trees URL: http://gameoftrees.org/
gitup URL: https://github.com/johnmehr/gitup
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
Contact: Warner Losh <imp@FreeBSD.org>
Contact: Ed Maste <emaste@FreeBSD.org>
Contact: FreeBSD-git mailing list
Contact: #gitcvt channel on EFnet
With the end of the third quarter of 2021, we are approaching the one-year
anniversary of the transition of the doc and src repositories from Subversi=
on
to Git. The 2021Q4 branch of the ports tree has been created, the third
quarterly branch created using Git.
During 2021Q3, repository hooks have been refined as problems were discover=
ed
and a few lingering references to Subversion were updated in the Committer=
=E2=80=99s
Guide. The Working Group continues to track progress on two
permissively-licensed Git-compatible tools: Gitup and Game of Trees. Gitup =
is a
small, dependency-free tool to clone and update git repositories. It is used
only to keep a local tree up-to-date, and has no support for local commits.
Game of Trees is a version control client that is compatible with Git
repositories. It provides a user interface and workflow that is distinct fr=
om
that of Git. It is in no way intended to be a drop-in replacement for Git, =
but
can be used to develop software maintained in a Git repository.
Gitup and Game of Trees are currently available as ports and packages. Futu=
re
work will evaluate them as candidates for the base system.
Remaining work includes continuing with refinements to Git documentation in=
the
Handbook, committing new and updated repository hooks for managing branch
content and commit mail, and surveying pre-commit hooks to use the CI clust=
er
to build release artifacts. Priorities have been set for the next working g=
roup
tasked with refining our Git workflow. The first meeting will be in
mid-October.
Sponsor: The FreeBSD Foundation (in part)
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
LLDB Debugger Improvements
Links:
Moritz Systems Project Description URL: link:https://www.moritz.systems/blo=
g/
freebsd-kgdb-support-in-lldb/
Progress Report 1 URL: https://www.moritz.systems/blog/
improving-gdb-protocol-compatibility-in-lldb/
Progress Report 2 URL: https://www.moritz.systems/blog/
improving-gdb-register-model-compatibility-in-lldb/
Git Repository URL: https://github.com/moritz-systems/llvm-project
Contact: Kamil Rytarowski <kamil@moritz.systems>
Contact: Micha=C5=82 G=C3=B3rny <mgorny@moritz.systems>
According to the upstream description, "LLDB is a next generation,
high-performance debugger. It is built as a set of reusable components which
highly leverage existing libraries in the larger LLVM Project, such as the
Clang expression parser and LLVM disassembler."
FreeBSD includes LLDB in the base system. At present, it has some limitatio=
ns
compared to the GNU GDB debugger, and does not yet provide a complete
replacement. This project spans from July 2021 to January 2022 and aims to =
make
LLDB suitable for debugging FreeBSD kernels.
The work so far was focused on improving the compatibility between LLDB and
other servers implementing the GDB Remote Protocol. Multiple bugs were fixed
that limited LLDB=E2=80=99s functionality when interfacing with GNU GDB=E2=
=80=99s gdbserver and
QEMU=E2=80=99s gdbserver implementations. Support for debugging executables=
running on
gdbserver architectures other than amd64 was fixed, and was explicitly test=
ed
on arm64 and i386.
This effort also resulted in adjusting lldb-server=E2=80=99s implementation=
to conform
better to the standard set by GDB. Thanks to these improvements, the LLDB
client needs to provide less divergent code paths depending on the server u=
sed,
and the single code path used is tested more thoroughly.
The work also involved improvements to the LLDB API and code deduplication,
that result in reducing the maintenance effort and lowering the bar for fut=
ure
contributions. The test coverage was improved, increasing the likelihood th=
at
any future problems will be detected before they affect users.
The introduced changes are expected to be shipped with LLDB 14.0.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Linux compatibility layer update
Contact: Dmitry Chagin, <dchagin@FreeBSD.org>
Contact: Edward Tomasz Napierala, <trasz@FreeBSD.org>
The goal of this project is to improve FreeBSD=E2=80=99s ability to execute=
unmodified
Linux binaries. The current support status of specific Linux applications is
being tracked on the Linux app status Wiki page.
The vdso(7) implementation was reworked. The futex(2) support was overhaule=
d to
make use of FreeBSD=E2=80=99s native umtx mechanism. It now supports
priority-inheritance futexes, in addition to fixing several bugs.
The rt_sigsuspend(2) and sigaltstack(2) syscalls are now supported on ARM64.
The faccessat2(2), clone3(2) system calls are now implemented. The
CLONE_CLEAR_RESETHAND option is now supported.
The prctl(2) syscall now supports PR_SET_NO_NEW_PRIVS.
The ptrace(2) syscall now supports PTRACE_GET_SYSCALL_INFO, which is a
prerequisite to support newer strace(1) versions.
There is ongoing work to make Linuxulator on arm64 on par with the amd64 on=
e;
right now it=E2=80=99s good enough for development work.
Sponsor: EPSRC (Edward=E2=80=99s work)
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Sound mixer improvements
Links:
GSoC 2021 project article URL: https://wiki.freebsd.org/
SummerOfCode2021Projects/SoundMixerImprovements
Contact: Christos Margiolis <christos@FreeBSD.org>
Contact: Hans Petter Selasky <hselasky@FreeBSD.org>
This project is an attempt to improve the capabilities of the OSS sound mix=
er
on FreeBSD. This means a new mixer(3) library, a complete rewrite of mixer(=
8),
and updates to sound(4).
Development started during Google Summer of Code 2021, but will likely cont=
inue
independently.
Applied changes:
=E2=80=A2 Implement OSS mixer (un)muting ioctls in sound(4) (commit 0f8da=
fb45859).
=E2=80=A2 Implement playback/recording mode sysctl (commit ed2196e5df0c).
=E2=80=A2 Implement mixer(3) and new version of mixer(8) (commit 903873ce=
1560).
Patches, comments and discussion are all welcome.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Base System OpenSSH Update
Links:
OpenSSH URL: https://www.openssh.com/
Announcement to freebsd-security@ URL: https://lists.freebsd.org/pipermail/
freebsd-security/2021-September/010473.html
Contact: Ed Maste <emaste@freebsd.org>
OpenSSH, a suite of remote login and file transfer tools, was updated from
version 7.9p1 to 8.7p1 in the FreeBSD base system.
FreeBSD base OpenSSH includes a number of local bug fixes, configuration
changes, and small features. As part of the work for this update, I submitt=
ed
some of these upstream and am preparing to do the same with the remaining
changes.
OpenSSH now supports FIDO/U2F devices, although additional work is required=
to
enable this in the FreeBSD base system. This includes importing a pair of
dependencies: libcbor, and libfido2. Within the next couple of months I exp=
ect
to import these, enable FIDO/U2F support, and update to OpenSSH version 8.8=
p1.
NOTE: OpenSSH 8.8p1 will disable the ssh-rsa signature scheme by default, so
some additional work is required for this next update. For more information
please see the Important note for future FreeBSD base system OpenSSH update
mailing list post.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Kernel
Updates to kernel subsystems/features, driver support, filesystems, and mor=
e.
Arm64 improvements
Links:
Pointer Authentication Review URL: https://reviews.freebsd.org/D31261
Contact: Andrew Turner <andrew@FreeBSD.org>
FreeBSD has been ported to the Arm Armv8-A Architecture Envelope Model (AEM=
),
an Armv8 architecture simulator. AEM is useful for Armv8 development and
testing, because it supports different configurations, including the abilit=
y to
enable or disable different extensions. As part of this work, the virtio le=
gacy
pci driver was fixed and ACPI resource parsing code was updated to work with
the ACPI tables that the model provides.
Libmd(4) has been updated to use the sha256 instructions when available. Th=
is
can lead to a large performance improvement when these instructions are
available. For example, on the Apple M1 under a hypervisor, the time to
calculate the sha256 of a 1GB file has decreased from a median of 3.46s to
0.5s.
Using an ifunc (indirect function) in a static binary is now supported. This
will allow different implementations of a function to be used depending on
which hardware it is being run on. Using an ifunc in dynamic binaries was
already supported.
The arm64 ID register definitions have been updated to match the Armv8.5
specification and other special registers have been updated to the Armv8.7
spec.
There have been numerous cleanups in decoding the arm64 ID registers in the
kernel to ensure we provide a consistent view of the hardware to userspace =
if
it reads the emulated ID register directly, or uses an ELF HWCAP value.
There has been early work on supporting the Arm Reliability, Availability, =
and
Serviceability (RAS) extension. The external aborts it may use are now hand=
led
in the kernel.
Support for the Arm Pointer Authentication extension is under review. This
extension allows programs to use new instructions that add a hash to unused
bits in a code or data pointer. They can then later check the hash in a way
that will either, depending on the CPU, create an invalid pointer or raise =
an
exception. This can be used to protect the function return address from bei=
ng
overwritten, making Return Oriented Programming (ROP) attacks difficult. The
change supports both userspace and kernel pointer authentication. It is wai=
ting
on debugger changes to be sent upstream so lldb can clear the hash when nee=
ded,
e.g., in a stack trace.
Work has started on supporting the Branch Target Identification extension. =
This
adds a flag to the page tables to disallow unintended targets from some bra=
nch
instructions. When a CPU branches to a new location from a register, the CPU
will check if the instruction is correct. This will protect, e.g., against a
function pointer being called when it points to the middle of a function. T=
he
kernel has been built with this feature enabled and tested in the Arm AEM.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
FreeBSD on Microsoft HyperV and Azure
Links:
Microsoft Azure article on FreeBSD wiki URL: https://wiki.freebsd.org/
MicrosoftAzure
Microsoft HyperV article on FreeBSD wiki URL: https://wiki.freebsd.org/Hype=
rV
Contact: Microsoft FreeBSD Integration Services Team <bsdic@microsoft.com>
Contact: freebsd-cloud Mailing List
Contact: The FreeBSD Azure Release Engineering Team <releng-azure@FreeBSD.o=
rg>
Contact: Wei Hu <whu@FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
The 13.0-RELEASE image on Azure Marketplace has been published.
The changes for building official images on Azure Marketplace, which fulfill
the requirements of Azure and FreeBSD cloud images like disk layout and UEFI
for Gen2 VM, along with some minor improvements like configurations to spee=
d up
booting, have been imported.
Above tasks are sponsored by The FreeBSD Foundation, with resources provide=
d by
Microsoft.
Microsoft Azure Network Adapter (MANA) is the new network adapter from
Microsoft which will be available in the Azure public cloud. It provides SR=
-IOV
NIC as a Virtual Function (VF) to guest OS running on Hyper-V. Wei has been
working on its driver for a while and committed in this quarter.
This task is sponsored by Microsoft.
Work in progress tasks:
=E2=80=A2 Build and publish gen2 vm image to Azure Marketplace
=E2=80=A2 Build and publish ZFS-based image to Azure Marketplace
=E2=80=A2 Upstream local modifications of Azure agent
=E2=80=A2 Update Azure agent port to the latest version
Open tasks:
=E2=80=A2 Update FreeBSD related doc at https://docs.microsoft.com
=E2=80=A2 Support FreeBSD in Azure Pipelines
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Improved amd64 UEFI boot
Contact: Konstantin Belousov <kib@FreeBSD.org>
UEFI BIOS on PC provides a much richer and more streamlined environment for
pre-OS programs, but also imposes some requirements on the programs that are
executed there, OS loaders in particular. One such requirement is that the
loader must coordinate its memory use with the BIOS, only using memory that=
was
allocated for it. Even after the loader handoff to the operating system ker=
nel,
there are still some memory regions that are reserved by the BIOS for diffe=
rent
reasons. Examples are runtime services code and data.
On the other hand, legacy/CSM BIOS boot works with memory completely
differently; there are known memory regions that hold BIOS data and must be
avoided. Otherwise, the memory is considered free for loader and OS to use.=
(Of
course it is not that straightforward, the definition of known regions is u=
p to
the vendor and there are a lot of workarounds there.)
The BIOS boot puts the kernel and preloaded data (like modules, memory disk,
CPU microcode update etc) at the contigous physical memory block starting at
2M. This algorithm goes back to how the i386 kernel boots.
Also, when preparing to pass control to the kernel, the loader creates very
special temporary mappings, where the low 1G of physical address space is
mapped 1:1 into virtual address space, and then repeated for each 1G until =
the
virtual memory end. The kernel knows about its physical location and the
temporary mapping, and constructs kernel page tables assuming that the phys=
ical
address of the text is at 2M.
This mechanism of loader to kernel handoff was left unchanged when the load=
er
gained support for the UEFI environment. The loader prepares kernel and
auxiliary preload data in a so-called staging area while UEFI boot services=
are
active, and after EFI_BOOT_SERVICES.ExitBootServices(), the temporary mappi=
ng
is activated and the staging area is copied at 2M.
An advantage at that time was that no changes to the kernel were needed. But
there are issues; the biggest is that memory at 2M might be not free for re=
use.
For instance, BIOS runtime code or data might be located there. Or there mi=
ght
be no memory at 2M at all. Or trampoline page table or code, or even some p=
arts
of the staging area overlapping with the 2M region where staging area is
copied. The outcome was a hard to diagnose boot time failure, typically a h=
ard
hang when the loader started the kernel.
Another limitation is the 1G transient mapping, which due to copying means =
that
the total size of preloaded data cannot exceed around 400M for everything,
including kernel, memory disks, and anything else. Also the code to grow the
staging area on demand was quite unflexible, only able to grow the staging =
area
in place.
The work described in this report allows the UEFI loader on amd64 to start =
the
kernel from the staging area without copying. Kernel assumptions about the
hand-off were explicitly identified and documented. The kernel only requires
the staging area to be located below 4G together with the trampoline page t=
able
(this is a consequence of CPU architecture requiring 32bit protected mode to
enter long mode), be 2M aligned and the whole low 4G mapped 1:1 at hand-off.
The kernel computes its physical address and builds kernel page tables
accordingly.
Making the kernel boot with staging area in-place required identifying all
places where the amd64 kernel had a dependency on its physical location. The
most complicated part was application processors startup, which required
rewriting initialization code, which we were able to streamline as result. =
In
particular, when an AP enters paging mode, it does so straight into the cor=
rect
kernel page table, without loading intermediate trampoline page table.
The updated loader automatically detects if the loaded kernel can handle
in-place staging area ('non-copying mode'). If needed, this can be overridd=
en
with the loader=E2=80=99s copy_staging command. For instance, 'copy_staging=
enable'
tells the loader to unconditionally copy the staging area to 2M regardless =
of
kernel capabilities (default is 'copy_staging auto'). Also, the code to grow
the staging area was made much more robust, allowing it to grow without
hand-tuning and recompiling the loader.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
ENA FreeBSD Driver Update
Links:
ENA README URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbs=
d/
ena/README
Contact: Michal Krawczyk <mk@semihalf.com>
Contact: Artur Rojek <ar@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>
ENA (Elastic Network Adapter) is the smart NIC available in the virtualized
environment of Amazon Web Services (AWS). The ENA driver supports multiple
transmit and receive queues and can handle up to 100 Gb/s of network traffi=
c,
depending on the instance type on which it is used.
Completed since the last update:
=E2=80=A2 Update ENA driver to v2.4.1
=E2=80=A2 Introduce full kernel RSS API support
=E2=80=A2 Allow reconfiguration of the RSS indirection table and hash key
=E2=80=A2 Support netmap on the c6gn/m6i AWS instance types
=E2=80=A2 Fix race between detach function and some of the procedural sys=
ctl nodes
=E2=80=A2 Add new statistics counters
=E2=80=A2 Improve safety of the reset handling routine
=E2=80=A2 Avoid mbuf collapse on Tx path for the LLQ condition
Work in progress:
=E2=80=A2 Prototype the driver port to the iflib framework
=E2=80=A2 MFC the ENA v2.4.1 driver to the FreeBSD 11/12/13-STABLE branch=
es
=E2=80=A2 Add IPv6 L4 checksum offload support to the driver
Sponsor: Amazon.com Inc
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Hole-punching support on FreeBSD
Links:
Commit 0dc332bff200 URL: https://cgit.freebsd.org/src/commit/?id=3D
0dc332bff200c940edc36c4715b629a2e1e9f9ae
Commit 9e202d036dd6 URL: https://cgit.freebsd.org/src/commit/?id=3D
9e202d036dd6f38ce0f578aa2086ebc358315bab
Commit 5ee2c35751ef URL: https://cgit.freebsd.org/src/commit/?id=3D
5ee2c35751ef5d131222423bf3e25073f997c337
OpenZFS Pull Request #12458 URL: https://github.com/openzfs/zfs/pull/12458
Contact: Ka Ho Ng <khng@FreeBSD.org>
Hole-punching functionality allows turning a contiguous range of bytes into=
a
hole for a given file. File systems supporting hole-punching may deallocate=
the
file system space from the given file. One use case for the functionality i=
s to
convert TRIM/UNMAP/DEALLOCATE requests from a virtual machine guest to
hole-punching calls on the host=E2=80=99s side, thus allows reclamation of =
file system
space when unneeded by the guest.
A set of APIs and KPIs are added that developers can call to invoke
hole-punching on a given file, if the underlying file system exposes that
functionality. For file systems not supporting hole-punching there is a
fallback implementation in the kernel which does zero-filling instead. Besi=
des
the APIs and KPIs addition, the truncate(1) utility is expanded by adding a=
-d
flag to support invoking hole-punching as well.
At the time of writing this report, OpenZFS and tmpfs are the file systems =
that
support hole-punching.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Intel Networking on FreeBSD
Links:
DPDK URL: https://github.com/DPDK/dpdk/tree/main/drivers/net
Contact: Intel <freebsd@intel.com> Contact: Kevin Bowling <kbowling@FreeBSD=
=2Eorg
>
lem(4)/em(4)/igb(4) and ix(4) were updated with various common code
improvements obtained from DPDK. There have also been several bug fixes from
the community:
=E2=80=A2 lem(4)/em(4)/igb(4) changes
=E2=80=A2 ix(4) changes
Intel has updated ixl(4) with various improvements:
=E2=80=A2 ixl(4): Fix 2.5 and 5G speeds reporting and update shared code
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Intel Wireless driver support
Links:
iwlwifi status FreeBSD wiki page URL: https://wiki.freebsd.org/WiFi/Iwlwifi
Contact: Bjoern A. Zeeb <bz@FreeBSD.ORG>
The Intel Wireless driver update project aims to bring support for newer
chipsets along with mac80211 LinuxKPI compat code. The dual-licensed Intel
driver code was ported in the past for the iwm(4) native driver; using the
LinuxKPI compat framework allows us to use the driver directly, with only v=
ery
minor modificationsi that we hope will be incorporated into the original
driver.
After the initial snapshot at the end of June, another snapshot was release=
d in
early September. Thank you to everyone testing on CURRENT and reporting bac=
k.
While we keep updating the driver and all the compat code needed for that, =
the
focus now is on stability and adding support for newer 802.11 standards. The
driver is currently set to 11a/b/g and 11n will be next before we look at 1=
1ac.
We are currently trying to get as much into FreeBSD as is possible and makes
sense. Full integration into FreeBSD main is waiting for a policy update.
For the latest state of the development or code, please follow the referenc=
ed
wiki page or the freebsd-wireless mailing list. We expect to release another
snapshot soon and hope to also support stable/13 again with that one.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Microchip LAN743x mgb(4) Device Driver
Links:
Commit adding mgb(4) driver URL: https://cgit.freebsd.org/src/commit/?id=3D
8890ab7758b8a03a68f3fe596f6a5446921631a5
Commit connecting mgb(4) module to the build URL: https://cgit.freebsd.org/=
src/
commit/?id=3D543df609072fe49079c36d6bee510e1645edde3a
Contact: Ed Maste <emaste@freebsd.org>
Microchip=E2=80=99s LAN7430 and LAN7431 are PCIe Gigabit Ethernet interface=
ICs, with
an integrated PHY (LAN7430) or RGMII interface (LAN7431).
In 2019 Gerald ND Aryeetey, a FreeBSD Foundation intern, developed the mgb(=
4)
driver for these devices. It was added to the tree but not yet connected to=
the
build. Since it was committed a number of sweeping iflib changes were made,
which included updates to mgb(4).
I have addressed some outstanding issues in the driver, and have now added =
the
module to the build. It is available in -CURRENT snapshot images. The drive=
r is
functional, although some additional work is still needed. In particular,
configuration of the Receive Filtering Engine, and VLAN and RX/TX checksum
offload are required. Caveats and notes are described in the man page.
I am very interested in feedback and test results.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Fixes for msdosfs_rename VOP
Contact: Konstantin Belousov <kib@FreeBSD.org>
Contact: Peter Holm <pho@FreeBSD.org>
Our msdosfs(5) implementation is old code and has a relatively large legacy
cost. In particular, even though it got fine-grained locking and miscellane=
ous
bugfixes over time, sometimes a serious issue is found in it.
Recently trasz@ found that msdosfs rename can be easily deadlocked. Further
examination of rename code revealed a lot of issues with locking, potential=
use
after free, and filesystem structure corruption.
As part of the update, locking in the msdosfs rename code was reworked. We =
need
to lock up to four vnodes, and check one path to ensure that rename does not
create circular parent relations between directories. For that, the locking
procedure was copied from UFS rename, where all vnodes except the first are
locked in try-mode. Lockless relockup was added to msdosfs and the directory
path checker was changed to non-blocking mode.
During this work, all known issues were fixed and msdosfs passes the enhanc=
ed
stress2 suite of tests.
Sponsor: The FreeBSD Foundation (kib=E2=80=99s contributions)
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
OpenZFS RAIDZ Expansion update
Links:
OpenZFS Pull Request URL: https://github.com/openzfs/zfs/pull/12225
video from 2021 FreeBSD Developer Summit URL: https://www.youtube.com/watch=
?v=3D
yF2KgQGmUic
slides from 2021 FreeBSD Developer Summit URL: https://docs.google.com/
presentation/d/1FeQgEwChrtNQBHfWSNsPK3Y53O5BnPh3Cz5nRa5GAQY/view
news article from Ars Technica URL: https://arstechnica.com/gadgets/2021/06/
raidz-expansion-code-lands-in-openzfs-master/
Contact: Matthew Ahrens <matt@mahrens.org>
Project
This feature allows disks to be added one at a time to a RAID-Z group,
expanding its capacity incrementally. This feature is especially useful for
small pools (typically with only one RAID-Z group), where there isn=E2=80=
=99t
sufficient hardware to add capacity by adding a whole new RAID-Z group
(typically doubling the number of disks).
FreeBSD=E2=80=99s ZFS implementation comes from the OpenZFS project. This w=
ork will be
integrated into the OpenZFS repo, with support for FreeBSD and Linux.
Status
The work is functionally complete, and a pull request has been opened.
Remaining Work
Remaining work includes some code cleanup, design documentation, addressing
test failures.
We also need to solicit code reviewers and respond to code review feedback.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
RFC1191 support in IPSEC tunnels
Contact: Wojciech Macek <wma@semihalf.com>
The Semihalf team has been working on providing support for RFC1191 in IPSEC
tunnels. This allows the PMTUD algorithm to dynamically discover the maximum
path MTU the tunnel can handle and update tunnel parameters accordingly. As=
a
result, a better performance can be observed as there is no need for packet
fragmentation.
The newly introduced rework includes:
=E2=80=A2 NEEDFRAG support for IPSEC commit d9d59bb1af1
=E2=80=A2 PMTUD for IPv4 IPSEC over IPv6 link commit 4f3376951d70
=E2=80=A2 Security fix as per RFC1191 specs commit b4220bf387e6
=E2=80=A2 PMTUD support for IPv6 tunnel commit 9dfc8606eb80
Sponsor: Stormshield
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Realtek rtw88 support
Links:
Rtw88 status FreeBSD wiki page URL: https://wiki.freebsd.org/WiFi/Rtw88
Contact: Bjoern A. Zeeb <bz@FreeBSD.ORG>
This project aims to bring in support for Realtek=E2=80=99s rtw88 chips.
With the growing LinuxKPI compatibility code an initial port of the Realtek
rtw88 driver was done. This gives us support for a second driver after Intel
and helps to validate and grow the LinuxKPI code base. Changes and overall =
time
needed to get the driver compiling were very little. At the moment we are
seeing DMA issues which prevent most people from loading firmware or using =
the
driver later on.
Thanks to everyone who was brave enough to give this initial code drop a tr=
y.
While this is a leasure time project we would love to better support Realtek
wireless devices and will keep working on this.
For the latest state of the development or code, please follow the referenc=
ed
wiki page or the freebsd-wireless mailing list.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Marvell SDHCI improvements and ACPI support
Contact: Bartlomiej Grzesik <bag@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>
The Semihalf team was working on the ACPI support for the Marvell Octeon TX2
CN913x and Armada 7k/8k SoCs' SD/MMC controller, as well as extending and
improving the related generic parts of the FreeBSD OS.
Applied changes:
=E2=80=A2 Add ACPI_GET_PROPERTY to access Device Specific Data (DSD) comm=
it
b91fc6c43a81
=E2=80=A2 Introduce generic device_get_property/device_has_property, whic=
h allow to
obtain DT/ACPI properties with the same generic code commit 3f9a00e3b577
=E2=80=A2 Switch mmc_helper to device_* API, to access the controller des=
cription
from DT/ACPI in a unified way commit 8a8166e5bcfb
=E2=80=A2 Add ACPI support for the sdhci_xenon driver commit d78e464d2330=
and commit
adbce5ff747b
=E2=80=A2 Apply other minor improvements and fixes related to properties =
parsing in
core mmc code and sdhci_xenon driver.
Sponsor: Semihalf
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Stack gap handling improvements
Contact: Dawid Gorecki <dgr@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>
Stack gap is a feature that randomizes the stack address by creating a rand=
om
sized gap between the top of stack and strings area. The current implementa=
tion
of this mitigation can cause issues for some applications e.g. Firefox (
PR239873). Until now the only workaround for this problem was disabling the
stack gap for those programs, as is done for ntpd.
Semihalf team has been working on fixing the root causes of stack gap relat=
ed
problems.
One of the issues could be observed when setting the stack limit to a low v=
alue
with setrlimit(2). Since the stack gap size can be significant, and the pro=
cess
had no knowledge of how large the stack gap is, this caused programs to abo=
rt
immediately after returning from a syscall due to the stack extending past =
the
limit. The fix involves increasing the soft resource limit (rlim_cur) by the
size of stack gap during the setrlimit(2) call. This fixes the issue with n=
tpd
without disabling the stack gap entirely. The patch is currently under revi=
ew.
The second identified problem is related to the way the thread stacks are
handled. Thread stacks are calculated using the kern.usrstack sysctl, which
should point to the beginning of the stack. This sysctl returned a constant
value depending on the ABI and the presence of the stack gap was ignored. A=
new
sysctl was introduced, and the thread library was updated to use it
accordingly. Patches D31897 and D31898 are currently under discussion. These
fix the issues with Firefox and Thunderbird not starting with the stack gap
feature enabled.
Sponsor: Stormshield
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
syzkaller on FreeBSD
Contact: Mark Johnston <markj@FreeBSD.org> Contact: Michael Tuexen <
tuexen@FreeBSD.org>
syzkaller is a coverage-guided operating system kernel fuzzer. See the
syzkaller entry in the 2019q1 quarterly report for an introduction to
syzkaller.
In the past quarter we made a concerted effort to shrink the backlog of rep=
orts
=66rom the public syzbot instance. A number of long-standing locking bugs i=
n the
socket subsystem have been fixed, and the SCTP protocol implementation has =
seen
many bug fixes as well. Beyond that, many bugs in various other kernel
subsystems have been fixed and the backlog has become substantially smaller
over the past quarter. As a direct result of this effort, we have been able=
to
identify regressions more easily and fix bugs closer to the time of
introduction. Work is still ongoing to further shrink the backlog.
KASAN (Kernel Address SANitizer) was enabled in the default kernel
configuration used by syzbot, which has greatly enhanced our ability to
root-cause and fix kernel bugs. See the kernel-sanitizers entry in the 2021=
q2
quarterly report for an introduction to KASAN and KMSAN. KASAN helps ensure
that memory safety bugs manifest more deterministically, improving syzkalle=
r=E2=80=99s
ability to find reproducers and simplifying debugging.
A KMSAN (Kernel Memory SANitizer) port was committed to FreeBSD=E2=80=99s m=
ain branch
in August. Some initial work has been done to make it usable by syzkaller
(mainly, kcov(4) required several small modifications to work in a
KMSAN-enabled kernel), and a number of bugs were found and fixed using priv=
ate
syzkaller instances.
Goals for the next several months include:
=E2=80=A2 Addition of a KMSAN target in syzbot.
=E2=80=A2 Further reduction in the syzbot backlog.
=E2=80=A2 Upstreaming syzkaller patches to support fuzzing of the Linuxul=
ator.
=E2=80=A2 Fuzzing using ZFS-based VM images.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Using OCF in WireGuard
Contact: John Baldwin <jhb@FreeBSD.org>
I have looked at updating the data path crypto in the upstream WireGuard dr=
iver
to use the in-kernel OpenCrypto Framework for the data path. Data packets s=
ent
over a WireGuard tunnel are encrypted with the Chacha20-Poly1305 AEAD ciphe=
r.
Unlike TLS and IPsec, Wireguard uses an 8 byte nonce rather than a 12 byte
nonce with this cipher.
To date, most of the work has focused on updating OCF to better support
multiple nonce (and tag/MAC) lengths for a given cipher. I resurrected an o=
ld
branch I had previously started on for this aimed at supporting all of the
AES-CCM NIST KAT vectors many of which use non-default nonce and tag length=
s. I
have refined the approach to better fit the existing OCF model where nonce =
and
MAC lengths are properties of a session (similar to key lengths). (My earli=
er
branch had made the nonce length a property of individual operations instea=
d.)
Mostly this entailed extending the /dev/crypto interface to permit setting
these parameters for a session. Existing tests for OCF run in userland and =
use
the /dev/crypto interface including both the cryptocheck utility and the NI=
ST
KAT vector tests.
Building on these framework changes, I have extended the existing
Chacha20-Poly1305 cipher in OCF to support both 8 and 12 byte nonces includ=
ing
in the accelerated ossl(4) driver. I also have a patch against the upstream
WireGuard FreeBSD driver to make use of this for the dataplane and have
verified that it interoperates with the stock WireGuard driver.
Future work will focus on upstreaming the OCF changes as well as additional
review of the upstream WireGuard driver.
Sponsor: The FreeBSD Foundation
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Intel Volume Management Device driver update
Links:
main branch commit URL: https://cgit.freebsd.org/src/commit/?id=3D
7af4475a6e31202a865b1dd3727018659b44470f
Contact: Alexander Motin <mav@FreeBSD.org>
Intel VMD is used by Intel=E2=80=99s VROC (Virtual RAID on chip) to isolate=
parts of
PCI tree, including NVMe devices, behind one or several VMD devices. Each V=
MD
device has 3 memory windows to access config space and memory of its child
devices, plus some number (I saw 19 and 33) of MSI-X interrupt vectors to
forward their MSI and MSI-X interrupts through.
The vmd(4) driver was first introduced in FreeBSD 13.0, but was found to ha=
ve a
number of problems addressed in this update: - The VMD device was made to l=
ook
more like a regular PCI-to-PCI bridge, implementing the regular pcib(4)
interface and using the standard pci(4) bus driver on top. - Memory and bus
numbers resource management was rewritten to properly handle resource reque=
sts
using memory windows of the VMD device. - Interrupt handling was completely
rewritten to distribute child interrupts among available VMD MSI-X vectors,
setting them up to be handled via standard OS mechanisms with minimal overh=
ead
instead of custom dispatching via taskqueue. Due to the limited number of
vectors and routing abilities of VMD, children are limited to only one MSI =
or
three MSI-X vectors per device to avoid sharing. The limits can be tuned,
depending on the number of VMD vectors and children. To improve the flexibi=
lity
the nvme(4) driver was made to work with a limited number of vectors, start=
ing
=66rom just one MSI/MSI-X if needed.
Sponsor: iXsystems, Inc.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Ports
Changes affecting the Ports Collection, whether sweeping changes that touch
most of the tree, or individual ports themselves.
Improve CPE Data in the Ports Collection
Links:
chkcpe URL: https://github.com/decke/chkcpe
CPE on wiki.freebsd.org URL: https://wiki.freebsd.org/Ports/CPE
CPE Dictionary URL: https://nvd.nist.gov/products/cpe
Contact: Bernhard Froehlich <decke@FreeBSD.org>
Common Platform Enumeration (CPE) is an open standard for naming hardware a=
nd
software components. Originally developed by MITRE, it is now maintained by
NIST under the aegis of the National Vulnerability Database. A CPE URI uniq=
uely
identifies a device or program by its vendor, product name, version, revisi=
on,
etc. NIST maintains the official CPE dictionary which lists CPE names for a=
ll
software versions that have ever been mentioned in an advisory.
In the FreeBSD Ports Collection it has been possible to annotate CPE data w=
ith
USES=3Dcpe since 2014 but only around 1000 ports out of 30.000 did use it. =
This
is why a script called chkcpe was created to validate existing CPE data and
find new possible matches automatically.
Why do we need it?
It allows comparing CPE URIs for installed packages against published
vulnerability data and will give us a much better and more complete pkg aud=
it.
As a side effect we will also have a better overview of vulnerable ports in=
the
FreeBSD Ports Collection that need to be patched or updated.
How can I help?
In this phase there is no easy possibility for port maintainers to help.
Creating separate PRs only to add CPE data does not make sense because the
overhead is very high. This is why I did spend some time on chkcpe to impro=
ve
the review and commit workflow. Right now review and commit consumes around=
3
minutes per port.
If you are a Ports Committer and want to help please get in touch!
Open Tasks
=E2=80=A2 Review remaining reports (~1800) and update ports when appropri=
ate
=E2=80=A2 Improve matching quality to find more possible matches
=E2=80=A2 Support using CPE data in pkg audit
=E2=80=A2 Scan for vulnerable ports in the Ports Collection
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
FreeBSD Erlang Ecosystem Ports update
Links:
FreeBSD Erlang wiki URL: https://wiki.freebsd.org/Erlang
Erlang/OTP language URL: https://erlang.org/
Elixir language URL: https://elixir-lang.org/
Gleam language URL: https://gleam.run/
Contact: FreeBSD Erlang mailing list <erlang@FreeBSD.org>
The Erlang runtime system, commonly known as the BEAM, provides a runtime t=
hat
is used by a number of programming languages and applications in the FreeBSD
ports collection.
Earlier this year, both Elixir and Erlang runtimes were brought up to date,=
but
as separate ports, to enable porters and users to test applications side by
side.
In Q3, the current runtimes have been brought across as defaults - this mea=
ns
that lang/elixir and lang/erlang are running as the latest releases of these
superb programming languages and runtimes.
Older releases of lang/erlang-runtime{21,22,23} are still available as port=
s.
The very old releases prior to OTP20 have been removed from the ports tree,=
as
they are no longer supported upstream either.
Only newer OTP releases include the updated SSL application that will corre=
ctly
validate cross-signed certificates, as used in Let=E2=80=99s Encrypt=E2=80=
=99s upcoming root
certificate deprecations.
Further details on these changes are well documented at Erlang/OTP impact of
DST Root CA X3 expiration and DST Root CA X3 expiration update
All of the NIF driver related ports that pull in other FreeBSD ports tree
dependencies have been updated to match the newer lang/erlang release, and a
number of ports that are not being updated in their upstream community, have
therefore been marked as broken.
The Erlang team is planning to:
=E2=80=A2 remove the deprecated OTP20 and OTP21 runtimes in 2021Q4
=E2=80=A2 remove ports directly dependent on erlang- and elixir- language=
s, where
they are more commonly installed via mix and rebar3 tools, the standard
community build tool chain.
Additional testing and community contributions are welcome; please reach ou=
t on
the mailing list, especially if you are able to help testing of specific po=
rt
updates.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
KDE on FreeBSD
Links:
KDE FreeBSD URL: https://freebsd.kde.org/
KDE Community FreeBSD URL: https://community.kde.org/FreeBSD
Contact: Adriaan de Groot <kde@FreeBSD.org>
The KDE on FreeBSD project aims to package almost all of the software from =
the
KDE Community for the FreeBSD ports tree. The software includes a full desk=
top
environment called KDE Plasma (for both X11 and Wayland) and hundreds of
applications that can be used on any FreeBSD machine.
The KDE team (kde@) is part of desktop@ and x11@ as well, building the soft=
ware
stack to make FreeBSD beautiful and usable as a daily-driver graphics-based
desktop machine.
Q3 is summer in the northern hemisphere: bicycles were biked and mountains =
were
hiked and the team was psyched to chase the software we like. And there were
plenty of things to chase:
=E2=80=A2 Three CMake releases landed, ending up with CMake 3.21.3 and li=
bpkg support
that once again works with CPack to produce FreeBSD packages.
=E2=80=A2 Monthly releases of KDE Frameworks, KDE Plasma, and KDE Gear ke=
pt the
exp-runs going. kde@ would like to thank Antoine for overseeing our many
exp-run requests.
=E2=80=A2 The KDE Plasma applications for monitoring the state of the
system=E2=80=89=E2=80=94=E2=80=89ksysguard, Plasma system monitor, and =
supporting
libraries=E2=80=89=E2=80=94=E2=80=89received a number of updates. FreeB=
SD support code has been
merged upstream.
=E2=80=A2 Across all of the Qt and KDE packages, an attempt was made to r=
educe how
"heavy" the dependencies are. In general this means removing developer-=
only
dependencies, avoiding the installation of test-code, etc. The underlyi=
ng
idea is to cut down the size of the installation when specific ports are
installed, while preserving the "developer batteries included" characte=
r of
the meta-ports.
=E2=80=A2 A runtime-incompatibility was introduced by MySQL 5.7.34, which=
affected
KDE=E2=80=99s personal-information-management applications and email. T=
his was
patched in the Qt ports.
=E2=80=A2 The mixer application in KDE Plasma now defaults to using pulse=
audio.
=E2=80=A2 accountsservice was updated, and then patched with code from Op=
enBSD.
=E2=80=A2 kdiff3 was updated to 1.9.3, now with upstream patches for some=
corner
cases originally reported on FreeBSD.
=E2=80=A2 kimageannotator and ksnip were updated, for nicer screenshots.
=E2=80=A2 kraft, the small-business support application, was updated to v=
ersion 0.97.
=E2=80=A2 krita had an upstream release to address a performance issue, w=
hich then
landed in FreeBSD.
=E2=80=A2 kstars, the interactive planetarium and sky-watching applicatio=
n, was
updated to 3.5.5.
=E2=80=A2 latte-dock, used as an alternative launcher, updated to 0.10.2.
=E2=80=A2 libphonenumber, Google=E2=80=99s library for dealing with all t=
he ways phone
numbers are represented around the world, received multiple updates.
=E2=80=A2 maliit-framework, for on-screen keyboards, as added to the port=
s tree.
=E2=80=A2 pipewire was kept up-to-date. This is a next-generation video-h=
andling
framework, and is being increasingly used in Wayland environments for
efficient passing of video data between applications.
=E2=80=A2 poppler was updated to version 21.09, with significant speed im=
provements.
=E2=80=A2 sddm was made a little more robust when installed on its own, w=
ith xinitrc
support.
=E2=80=A2 math/eigen2 was removed.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
OpenSearch on FreeBSD
Links:
OpenSearch website URL: https://opensearch.org/
OpenSearch Community (unofficial) on Discord URL: link:https://discord.gg/
NmtQGAWY
Contact: OpenSearch Team <opensearch@FreeBSD.org>
OpenSearch is a community-driven, open-source search and analytics suite
derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2. It
consists of a search engine daemon, OpenSearch, and a visualization and user
interface, OpenSearch Dashboards. OpenSearch enables people to easily inges=
t,
secure, search, aggregate, view, and analyze data. These capabilities are
popular for use cases such as application search, log analytics, and more. =
With
OpenSearch people benefit from having an open-source product they can use,
modify, extend, monetize, and resell how they want.
After the release of OpenSearch 1.0.0, a FreeBSD OpenSearch team was create=
d to
coordinate the efforts for porting it to FreeBSD. Because ElasticSearch 7 a=
nd
Kibana 7 were already in the ports tree, we could rely on these ports to get
started quickly (kudos to the elastic@ team) and could focus on the parts t=
hat
already changed between the previous codebase and the fork. The ports have =
been
committed to the FreeBSD ports tree as textproc/opensearch and textproc/
opensearch-dashboards, and currently provide the latest 1.0.1 version of
OpenSearch.
Contributing
The ports have been thoroughly tested in our testing environments and on so=
me
production workloads, but we are actively looking for feedback from users. =
Feel
free to send us an email to report successes or failures.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Valgrind - Official Support for FreeBSD has Landed
Links
Valgrind Home Page URL: https://www.valgrind.org/
Valgrind News URL: https://www.valgrind.org/docs/manual/dist.news.html
Contact: Paul Floyd <pjfloyd@wanadoo.fr>
Valgrind is an instrumentation framework for building dynamic analysis tool=
s.
There are Valgrind tools that can automatically detect many memory manageme=
nt
and threading bugs, and profile your programs in detail. You can also use
Valgrind to build new tools.
With the release of version 3.18.0 of Valgrind, official FreeBSD support has
landed upstream. The two supported architectures are amd64 and i386 (x86).
The next step will be to update the FreeBSD port. This will involve switchi=
ng
=66rom code that was maintained out-of-tree for over 15 years to using the
official upstream tarball.
As Valgrind is closely tied to operating system details, obtaining official
FreeBSD support was the result of significant effort from dozens of develop=
ers.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Wine on FreeBSD
Links Wine home page
Contact: Gerald Pfeifer <gerald@FreeBSD.org> Contact: Damjan Jovanovic <
damjan.jov@gmail.com>
Wine allows running Windows programs on FreeBSD.
This quarter we focused on the wine-devel port that tracks mainline
development, followed half a dozen of version updates, simplified the port,
removed three options (SDL, VKD3D, VULKAN) which are unconditionally active
now, and fixed a number of portability issues.
These changes will make their way into the primary Wine port over the coming
weeks.
After working on our Wine ports for more than 21 years, maintaining for more
than 19 years, time has come to hand over the baton.
Luckily Damjan has stepped up and is going to look after wine-devel and
associated ports - thanks! Help with the following is still very welcome:
=E2=80=A2 WineHQ bug 50257
=E2=80=A2 FreeBSD Bugzilla 257533
=E2=80=A2 Maintain daily (or at least regular) test builds of upstream Wi=
ne. This
quarter this triggered two dozen fixes in support of FreeBSD upstream,
though usually it=E2=80=99s quite less effort.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Ztop
Links:
Repository
Port
Contact: Alan Somers <asomers@FreeBSD.org>
ztop is a new utility that displays ZFS datasets' I/O in real time, like to=
p,
but for ZFS. Unlike the existing zpool iostat, which only displays traffic =
at
the level of a pool, ztop displays it at the level of individual datasets.
Previously, there was no tool that could display this information. The
Prometheus Node Exporter can export it into Prometheus, from which it can be
viewed after the fact with various other tools, but that requires a large s=
tack
of software, and isn=E2=80=99t real-time.
I started ztop this quarter, and it is now fully functional. However, it has
revealed a performance problem within the kernel. In the presence of more t=
han
a few thousand datasets, the sysctl interface is too slow and ztop becomes
sluggish. Future work, if I have the time, will address that problem.
Sponsor: Axcient
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Miscellaneous
Objects that defy categorization.
-CURRENT compilation time analysis
Links:
Bug 257141 URL: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257141
Discussion on freebsd-current URL: https://lists.freebsd.org/archives/
freebsd-current/2021-September/index.html#msg511
Visual chart of buildworld by stages URL: https://people.freebsd.org/wosch/
build-time/buildworld/
Contact: Wolfram Schneider <wosch@FreeBSD.org>
Re-building FreeBSD from source takes a lot of CPU resources. Depending on =
your
machine it takes between 20min and several hours. At the end of `make
buildworld' we log the real time and which parameters are used for parallel
build. E.g.
time make -j $(sysctl -n hw.ncpu) buildworld > buildworld.log 2>&1
tail buildworld.log
>>> World build completed on Sat Sep 4 20:58:00 UTC 2021
>>> World built in 7235 seconds, ncpu: 3, make -j3
7235.61 real 20527.30 user 915.88 sys
The build process runs in several steps. It would be great to know which st=
ep
takes most of the time, and what are the effects of special build parameter=
s.
With a small patch in Aug 2021 we get this information now:
egrep '>>> stage| real ' buildworld.log
>>> stage 1.1: legacy release compatibility shims
0.28 real 0.18 user 0.10 sys
>>> stage 1.2: bootstrap tools
165.99 real 472.58 user 11.56 sys
>>> stage 2.1: cleaning up the object tree
21.47 real 36.96 user 14.14 sys
15.87 real 29.14 user 11.87 sys
>>> stage 2.3: build tools
2.42 real 3.79 user 0.62 sys
>>> stage 3: cross tools
9.92 real 18.49 user 1.75 sys
>>> stage 3.1: recording build metadata
0.07 real 0.01 user 0.06 sys
>>> stage 4.1: building includes
16.62 real 36.46 user 9.48 sys
>>> stage 4.2: building libraries
5440.89 real 15724.60 user 482.58 sys
>>> stage 4.3: building lib32 shim libraries
615.91 real 1654.77 user 164.58 sys
>>> stage 4.4: building everything
937.23 real 2540.06 user 205.47 sys
In this example, we spent most of the time in "stage 4.2: building librarie=
s",
77% of the CPU time and 75% of the real time. Now running the buildworld wi=
th
the parameter WITHOUT_TOOLCHAIN=3Dyes we get a 3.3x faster build. Instead o=
f 2h
it will be done in 36 minutes!
time make -j $(sysctl -n hw.ncpu) WITHOUT_TOOLCHAIN=3Dyes buildworld > buil=
dworld.log 2>&1
>>> World build completed on Fri Sep 17 12:31:41 UTC 2021
>>> World built in 2207 seconds, ncpu: 3, make -j3
2207.44 real 5710.83 user 676.16 sys
egrep '>>> stage| real ' buildworld.log
>>> stage 1.1: legacy release compatibility shims
0.35 real 0.20 user 0.16 sys
>>> stage 1.2: bootstrap tools
20.47 real 51.98 user 5.12 sys
>>> stage 2.1: cleaning up the object tree
20.92 real 34.45 user 13.57 sys
16.33 real 28.59 user 12.33 sys
>>> stage 2.3: build tools
2.59 real 3.90 user 0.86 sys
>>> stage 3: cross tools
10.46 real 18.62 user 2.35 sys
>>> stage 3.1: recording build metadata
0.07 real 0.03 user 0.05 sys
0.08 real 0.03 user 0.05 sys
>>> stage 4.1: building includes
15.50 real 33.03 user 9.29 sys
>>> stage 4.2: building libraries
750.31 real 1962.43 user 218.60 sys
>>> stage 4.3: building lib32 shim libraries
684.05 real 1780.35 user 213.22 sys
>>> stage 4.4: building everything
677.87 real 1787.13 user 186.82 sys
Using WITHOUT_TOOLCHAIN=3Dyes is fine as long as there are no major LLVM co=
mpiler
changes.
If you dislike this feature or suspect it is causing trouble you can disabl=
e it
with the variable TIME_ENV=3D""
Next task: get more detailed timing information for the long-running stages
(4.2, 4.3, 4.4)
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Third-Party Projects
Many projects build upon FreeBSD or incorporate components of FreeBSD into
their project. As these projects may be of interest to the broader FreeBSD
community, we sometimes include brief updates submitted by these projects in
our quarterly report. The FreeBSD project makes no representation as to the
accuracy or veracity of any claims in these submissions.
Gitlab 14.3 available
Links:
Gitlab 14.3 New Features URL: https://about.gitlab.com/releases/2021/09/22/
gitlab-14-3-released/
Contact: Matthias Fechner <mfechner@FreeBSD.org>
GitLab is the DevOps Platform. Bring velocity with confidence, security wit=
hout
sacrifice, and visibility into DevOps success.
Version 14.3 now available on FreeBSD.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
helloSystem
Links:
Documentation URL: https://hellosystem.github.io/
Contact: Simon Peter <probono@puredarwin.org>
Contact: #helloSystem on irc.libera.chat, mirrored to #helloSystem:matrix.o=
rg
on Matrix
What is helloSystem?
helloSystem is FreeBSD preconfigured as a desktop operating system with a f=
ocus
on simplicity, elegance, and usability. Its design follows the =E2=80=9CLes=
s, but
better=E2=80=9D philosophy.
Q3 2021 Status
=E2=80=A2 Version 0.6.0 of helloSystem has been published including many =
contributed
features and bugfixes
=E2=96=A1 Improved window management with KWin as the window manager
=E2=96=A1 Simplified Filer file manager
=E2=80=A2 Contributors have started to port the helloSystem desktop envir=
onment,
helloDesktop, to the FreeBSD Ports and Packages Collection (see changel=
og
at https://github.com/helloSystem/ISO/releases/tag/r0.6.0 for details)
Installable Live ISO images and a full changelog are available at https://
github.com/helloSystem/ISO/releases/tag/r0.6.0
Contributing
Help is wanted in a number of areas, especially in the areas of the FreeBSD
core OS and kernel, and Qt/C++.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
Containers & FreeBSD: Pot, Potluck & Potman
Links:
Pot on github URL: https://github.com/pizzamig/pot
Potluck Repository & Project URL: https://potluck.honeyguide.net/
Potluck on github URL: https://github.com/hny-gd/potluck
Potman on github URL: https://github.com/grembo/potman
Contact: Luca Pizzamiglio (Pot) <pizzamig@freebsd.org>
Contact: Stephan Lichtenauer (Potluck) <sl@honeyguide.eu>
Contact: Michael Gmelin (Potman) <grembo@FreeBSD.org>
Pot is a jail management tool that also supports orchestration through Noma=
d.
In the last quarter, a new release 0.13.0 with a number of major new featur=
es
was made available.
Layered images allow a split into a foundation image component that only
changes seldom and that e.g. contains the basic jail and packages and a "le=
af"
image component on top with potentially small additions that might change m=
ore
frequently, like software built in a CI pipeline. Since it is not the compl=
ete
image that has to be rebuilt each time, image creation and distribution can=
be
sped up immensely.
Also a copy-out command has been included where the container state that was
initialised from within the container can be made persistent and reinjected
into additional containers afterwards.
Potluck aims to be to FreeBSD and Pot what Dockerhub is to Linux and Docker=
: a
repository of Pot flavours and complete container images for usage with Pot=
and
in many cases Nomad.
Here we had a busy quarter. Not only did we commit a number of new images l=
ike
Nextcloud which can be deployed via Nomad, but the images used to build the
foundation of a virtual data center (Nomad, Consul and Vault) themselves ha=
ve
received major updates.
For these images, further improvements for even better security and reliabi=
lity
will likely be finished in the coming quarter.
Also, we are aware that right now the advantages of the container approach =
as
it is described in Virtual DC Part I/Part II/Part III are not yet obvious a=
nd
transparent enough and also no longer completely up to date, so we plan to
provide additional documentation as soon as we find the time to do so.
Potman aims to simplify building Pot images with Vagrant and VirtualBox bas=
ed
on the Potluck approach, e.g. as part of a DevOps workflow for software
development including testing and publishing them to a repository.
Here, not too much has happened over the last quarter but the current idea =
is
to incorporate additional features in the medium to longer term to modulari=
se
and simplify the image build definitions and then utilise Potman in the Pot=
luck
library build process.
As always, feedback and patches are welcome.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
WireGuard on FreeBSD Stabilizes, Eyes Upstreaming
Links
WireGuard URL:https://www.wireguard.com/
wireguard-freebsd codebase URL:https://git.zx2c4.com/wireguard-freebsd/
Contact: Jason A. Donenfeld <Jason@zx2c4.com>
WireGuard is a secure tunneling protocol that lives in the kernel.
For the last quarter, the wireguard-freebsd codebase has been quite stable =
and
complete. For a while, there were rapid-fire releases fixing issues, and a =
lot
of effort was made to track down every bug report on bugs.freebsd.org, IRC,=
the
mailing list, or elsewhere. But by now, the reports have dried up, and most=
ly
users come to IRC with questions on usage and integration, the usual types =
of
things associated with a stabler project. We also have automated CI now for
each commit, compiling and running a small smoke test on wireguard-freebsd=
=E2=80=99s
supported releases=E2=80=89=E2=80=94=E2=80=8912.1, 12.2, and 13.0. At some =
point, hopefully that small
smoke test will expand to include the larger battery of tests from Linux.
The wireguard-freebsd repository has been geared around being an out-of-tree
kmod, which is distributed in ports. But it is also organized to be eventua=
lly
upstreamed. To that end, the repository maintains two files: compat.h and
support.h. compat.h contains polyfills of code that exists in FreeBSD=E2=80=
=99s main
branch but does not exist in various older releases, with ifdefs for each of
the various releases we support. On the other hand, support.h contains code
that is not yet in FreeBSD=E2=80=99s main branch. The goal is to eventually=
move all
the code from support.h into compat.h, at which point, the repository will =
be
ready for upstreaming. As of writing, there=E2=80=99s basically only one re=
al function
left=E2=80=89=E2=80=94=E2=80=89sogetsockaddr=E2=80=89=E2=80=94=E2=80=89and =
then two convenience macros that need to be sent
upstream for consideration by ConcurrencyKit.
A significant aspect that isn=E2=80=99t in support.h, though, is the crypto=
graphic
primitives that the code uses. The files crypto.c and crypto.h contain bori=
ng C
"reference implementations" of ChaCha20Poly1305, XChaCha20Poly1305, Blake2s,
and X25519 (which is formally verified via MIT=E2=80=99s fiat-crypto projec=
t). These
four algorithms are used by the handshake path on very small inputs for
WireGuard=E2=80=99s key exchange, and will hopefully be making their way to=
sys/crypto/
in the FreeBSD tree as just ordinary functions. On the flip side, the datap=
ath
uses an entry point of ChaCha20Poly1305 that works on mbufs (which might be
rather large) and is performance critical. To that end, jhb@ has been impro=
ving
OCF for WireGuard=E2=80=99s particulars. The next step will then be moving =
our current
calls from chacha20poly1305_{en,de}crypt_mbuf in wg_noise.c to instead call=
out
to OCF, submitting crypto_buffer`s of type `CRYPTO_BUF_MBUF. This will
automatically benefit from Andy Polyakov=E2=80=99s optimized ChaPoly implem=
entations
that OCF has long since imported from OpenSSL.
When we make the move to OCF, it=E2=80=99s likely that the wireguard-freebs=
d repo as-is
will become "wireguard-freebsd-compat", which will be explicitly aimed at
backports to earlier FreeBSD releases for ports, while a new wireguard-free=
bsd
repository will be a whole FreeBSD tree, where we can work directly on
integration patches for upstream. That repository will also have an imported
version of the wg(8) utility from wireguard-tools, which I=E2=80=99ll be re=
licensing as
MIT.
I=E2=80=99m quite excited for the upcoming quarter and seeing how much of
wireguard-freebsd we=E2=80=99re able to land upstream.
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81
--66dsezfsnajsewcm
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQGTBAABCgB9FiEEDonNJPbg/JLIMoS6Ps5hSHzN87oFAmGSjzBfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDBF
ODlDRDI0RjZFMEZDOTJDODMyODRCQTNFQ0U2MTQ4N0NDREYzQkEACgkQPs5hSHzN
87rCJQf+L85vcCZetfg8jx9wmIkBX8z1Sy4P0k0JYAlDU64bdSorQfv9iehSapMd
jGKLOhD5oQjFht/2u4uM9bBYMDBe53JSSo9NUSdcjPDsHcwLbZPKbxNlSZ6GOX6U
UmAGftRiXLTIX2DNTeNQmtUzd/Os66/13RTaUlkAg2L9zHxX6ImXWk27EWpsR6Nn
oY4gXT4/px750uBdYPv1HxGxLpHIMy4E46T7UOcrc3UmbhN+lAf6fj+yNIx04ur5
IYgPv8vAFffaTjvF/YzPbPj9L5UJhCzTQed8IE8Xbsl4r+NKKmFRCk0Tb+IIUiWA
vCnZ8iOJWV9vWTOZgxMHWD/43oxwyQ==
=n3Dy
-----END PGP SIGNATURE-----
--66dsezfsnajsewcm--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20211115164744.chsesrxkz3bb4wct>