index | | raw e-mail
Links: sysutils/slurm-wlm URL: https://cgit.freebsd.org/ports/tree/sysutils/slurm-wlm/ net/ucx URL: https://cgit.freebsd.org/ports/tree/net/ucx/ sysutils/mpifileutils URL: https://cgit.freebsd.org/ports/tree/sysutils/ mpifileutils/ devel/libcircle URL: https://cgit.freebsd.org/ports/tree/devel/libcircle/ devel/lwgrp URL: https://cgit.freebsd.org/ports/tree/devel/lwgrp/ devel/dtcmp URL: https://cgit.freebsd.org/ports/tree/devel/dtcmp/ benchmarks/py-reframe-hpc URL: https://cgit.freebsd.org/ports/tree/benchmarks/py-reframe-hpc/ openucx/ucx#11354: UCS/TYPE portability fixes for non-glibc/Clang environments (merged) URL: https://github.com/openucx/ucx/pull/11354 openucx/ucx#11549: UCS/SYS portability fixes for non-Linux platforms (in review) URL: https://github.com/openucx/ucx/pull/11549 hpc/mpifileutils#664: Portability fixes for FreeBSD/non-Linux builds (in review) URL: https://github.com/hpc/mpifileutils/pull/664 Kavocado Monthly Status Reports – FreeBSD HPC notes URL: https://kavocado.net/reports/ Contact: Generic Rikka <rikka.goering@outlook.de> This report continues the FreeBSD HPC Ports Modernization initiative. Previous quarters focused on bringing the Slurm + PMIx + PRRTE + UCX stack up to date and filling gaps in the surrounding ecosystem. This quarter’s work centered on three things: keeping the core scheduler and communication libraries current against fast-moving upstream releases, upstreaming the FreeBSD portability fixes accumulated while maintaining these ports so the local patchsets keep shrinking, and adding a new port for MPI-parallel file utilities that are common on large HPC filesystems. Work completed • Updated sysutils/slurm-wlm twice this quarter, from 25.11.4 to 25.11.5 and then from 25.11.5 to 26.05.1, tracking the latest upstream releases. • Fixed several FreeBSD-specific runtime issues in net/ucx: hardened async thread state handling, corrected UCM relocation handling, fixed mm signal socket binding, and resolved other FreeBSD runtime portability issues. • Updated net/ucx from 1.20.0 to 1.20.1, and separately fixed a libucm early-init panic together with several gtest suite failures uncovered while validating the update. • Added sysutils/mpifileutils as a new port, along with its dependency stack: devel/libcircle, devel/lwgrp, and devel/dtcmp. These provide MPI-parallel file utilities (copy, remove, checksum, etc.) commonly used on large HPC filesystems. • Updated benchmarks/py-reframe-hpc twice this quarter to track upstream releases of the regression testing framework. • Upstreamed "UCS/TYPE: Portability fixes for non-glibc/Clang environments" to UCX, which was reviewed and merged. • Opened "UCS/SYS: portability fixes for non-Linux platforms" against UCX, covering hostname buffer sizing, MAC address retrieval via getifaddrs()/ AF_LINK, shared-memory error reporting, mmap/munmap-based reallocation, and cpuset_getaffinity()-based thread affinity queries; currently addressing review feedback. • Opened a portability PR against mpifileutils guarding Linux-only APIs for FreeBSD and other non-Linux builds; currently in review. • Collaborated with SchedMD on a Slurm configure-time probe for H5PTopen, used to detect the HDF5 high-level library, which was merged upstream. Work in progress • Addressing reviewer feedback on the open UCX and mpifileutils portability pull requests to get them merged upstream and reduce the FreeBSD ports' local patchsets further. • Continuing collaboration with SchedMD to upstream additional portability fixes discovered while maintaining sysutils/slurm-wlm on FreeBSD, including BSD socket-handling improvements. • Evaluating a proctrack/freebsd_reaper Slurm plugin design, made possible by recently landed FreeBSD kernel support for exposing reaper metadata in process information. • Continuing to track upstream Slurm, UCX, and PMIx/PRRTE releases closely to keep the FreeBSD ports current with minimal local patching. Future plans • Land the outstanding UCX and mpifileutils upstream pull requests, and continue identifying further portability issues to upstream rather than patch locally. • Prototype and submit the Slurm proctrack/freebsd_reaper plugin once the underlying kernel interface work is finalized. • Continue expanding the HPC software ecosystem available in the FreeBSD Ports Collection and reduce local patchsets across the stack wherever upstream acceptance is possible. • Document a reference Slurm + OpenMPI + PMIx + PRRTE + UCX deployment on FreeBSD to lower the barrier for new sites experimenting with FreeBSD in an HPC context. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Improve libvirt support for bhyve hypervisor Links: libvirt: Bhyve driver URL: https://libvirt.org/drvbhyve.html Contact: Roman Bogorodskiy <novel@FreeBSD.org> Completed work • libvirt/bhyve driver: □ virtio-console device support added. □ QEMU Guest Agent support added. □ Implemented a few features based on the QEMU Guest Agent, such as graceful reboot and shutdown, managing guest OS time, SSH keys, users passwords, etc. □ Added memory tuning and block I/O tuning based on rctl(8). • Work in progress on extend libvirt CI to test against FreeBSD-CURRENT snapshot VM images. Plans for the next quarter • Add support for boot order configuration. • Add support for TPM devices. • Complete suspend/resume support. • Continue implementing features based on the QEMU Guest Agent. • Continue extending CI, cover the new guest agent based features with new tests. Sponsor: The FreeBSD Foundationhome | help