Date: Sat, 11 Jul 2026 20:49:56 +0000 From: Dave Cottlehuber <dch@FreeBSD.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org Subject: git: 688a84db45 - main - containers: Add OCI setup and usage Message-ID: <6a52ac74.3646c.35322aa5@gitrepo.freebsd.org>
index | next in thread | raw e-mail
The branch main has been updated by dch: URL: https://cgit.FreeBSD.org/doc/commit/?id=688a84db457859f28d11cc733164a34bdd388121 commit 688a84db457859f28d11cc733164a34bdd388121 Author: Dave Cottlehuber <dch@FreeBSD.org> AuthorDate: 2026-07-11 20:30:51 +0000 Commit: Dave Cottlehuber <dch@FreeBSD.org> CommitDate: 2026-07-11 20:49:43 +0000 containers: Add OCI setup and usage Reviewed by: adrian Reviewed by: bcr Reviewed by: dfr Approved by: ziaee Sponsored by: SkunkWerks, GbmH Differential Revision: https://reviews.freebsd.org/D57862 --- .../en/books/handbook/containers/_index.adoc | 865 +++++++++++++++++++++ 1 file changed, 865 insertions(+) diff --git a/documentation/content/en/books/handbook/containers/_index.adoc b/documentation/content/en/books/handbook/containers/_index.adoc index 62c4d516a6..1c7b21bcb0 100644 --- a/documentation/content/en/books/handbook/containers/_index.adoc +++ b/documentation/content/en/books/handbook/containers/_index.adoc @@ -185,3 +185,868 @@ Use these if you want to test against what subsequent future releases of FreeBSD * `+freebsd-runtime:14.snap+` - always the latest build from 14-STABLE branch * `+freebsd-runtime:15.snap+` - always the latest build from 15-STABLE branch * `+freebsd-runtime:16.snap+` - always the latest build from 16-CURRENT branch + +[[containers-registries]] +== Official Registry Locations + +Architecture-independent images can be downloaded and directly imported into your own registry from the https://download.freebsd.org/releases[Official FreeBSD Releases] site, or alternatively, pulled from the FreeBSD project's https://hub.docker.com/u/freebsd[Docker Hub], or https://github.com/orgs/freebsd/packages[GitHub Container Registry]. + +Both https://hub.docker.com/_/alpine/[Alpine Linux] and the https://hub.docker.com/_/debian/[Debian] project publish tagged and mutable images, using similar approaches. + +[[containers-podman-intro]] +== Introduction to Podman + +The Podman suite comprises three main tools and a number of supporting ones: + +* https://podman.io/[podman] is a daemonless container engine using docker-compatible commands +* https://buildah.io/[buildah] generates OCI formatted container images +* https://github.com/containers/skopeo[skopeo] works with remote OCI image registries + +[[containers-preparing]] +== Preparing the System for Podman Containers + +[WARNING] +==== +If running commands over SSH, or in a similar remote session, use man:screen[1], man:tmux[1], or similar to avoid losing the connection while restarting the firewall. +==== + +If ZFS is not available, switch to UFS storage and skip the ZFS commands below: + +[source,shell] +---- +# sed -I .bak -e 's/driver = "zfs"/driver = "vfs"/' \ + /usr/local/etc/containers/storage.conf +---- + +Otherwise, create the container storage dataset: + +[source,shell] +---- +# zfs create -o mountpoint=/var/db/containers zroot/containers +# zfs snapshot zroot/containers@empty +---- + +The File Descriptor filesystem is required: + +[source,shell] +---- +# mount -t fdescfs fdesc /dev/fd +---- + +Podman uses FreeBSD's packet filter to forward container ports to the host network: + +[source,shell] +---- +# test -c /dev/pf || kldload pf +# sysctl net.pf.filter_local=1 +---- + +Amend `+/etc/sysctl.conf+` and `+/etc/fstab+` as appropriate, to make these changes permanent. + +[[containers-installing]] +== Installing Podman + +Only the `+sysutils/podman-suite+` meta-package is required, but if the additional package:emulators/qemu-user-static[] package is installed, it becomes possible to build or test images for other architectures such as `+arm64+` on an `+amd64+` host. + +[source,shell] +---- +# pkg install -r FreeBSD -y podman-suite emulators/qemu-user-static +---- + +Integrate changes from `+/usr/local/etc/containers/pf.conf.sample+` into `+/etc/pf.conf+`, setting egress macros appropriately, then restart the firewall: + +[source,shell] +---- +# service pf restart +---- + +The packages install a number of template configuration files, none of which need to be edited immediately. +Review and amend these as needed: + +[source,shell] +---- +# pkg list buildah podman conmon \ + ocijail containers-common \ + containernetworking-plugins \ + | grep /etc/ +/usr/local/etc/containers/containers.conf.sample +/usr/local/etc/containers/policy.json.sample +/usr/local/etc/containers/registries.conf.sample +/usr/local/etc/containers/storage.conf.sample +/usr/local/etc/containers/pf.conf.sample +---- + +[[containers-terminal-tour]] +== Importing and Running Containers + +[NOTE] +==== +At present, all Podman containers on FreeBSD must run as root, as FreeBSD jails require this. +==== + +With the necessary tools and firewall rules in place, the officially published images can be fetched from https://download.freebsd.org/releases/OCI-IMAGES/[FreeBSD Releases], for https://download.freebsd.org/releases/OCI-IMAGES/{rel-latest}-RELEASE/aarch64/Latest/[aarch64] or https://download.freebsd.org/releases/OCI-IMAGES/{rel-latest}-RELEASE/amd64/Latest/[amd64] as appropriate. + +[[containers-importing]] +=== Importing FreeBSD OCI Images + +It is simplest to pull images directly from a public container registry, but for the strongest chain of trust, download them from https://download.freebsd.org/releases[FreeBSD.org] directly, and verify the checksums against the PGP-signed release announcement. + +[source,shell,subs="attributes"] +---- +# export OCIBASE=https://download.freebsd.org/releases/OCI-IMAGES/{rel-latest}-RELEASE/amd64/Latest +# podman load -i=$OCIBASE/FreeBSD-{rel-latest}-RELEASE-amd64-container-image-static.txz +Getting image source signatures +Copying blob 44defc6cee2e done | +Copying config 5921c62e76 done | +Writing manifest to image destination +Loaded image: localhost/freebsd-static:{rel-latest}-RELEASE-amd64 + +# podman load -i=$OCIBASE/FreeBSD-{rel-latest}-RELEASE-amd64-container-image-dynamic.txz +Getting image source signatures +Copying blob 6054dffc3887 done | +Copying config 33427d59f9 done | +Writing manifest to image destination +Loaded image: localhost/freebsd-dynamic:{rel-latest}-RELEASE-amd64 + +# podman load -i=$OCIBASE/FreeBSD-{rel-latest}-RELEASE-amd64-container-image-runtime.txz +Getting image source signatures +Copying blob dcbbfcb598fe done | +Copying config 90c4936754 done | +Writing manifest to image destination +Loaded image: localhost/freebsd-runtime:{rel-latest}-RELEASE-amd64 + +# podman load -i=$OCIBASE/FreeBSD-{rel-latest}-RELEASE-amd64-container-image-notoolchain.txz +Getting image source signatures +Copying blob a16c94176cc3 done | +Copying config 086572f35c done | +Writing manifest to image destination +Loaded image: localhost/freebsd-notoolchain:{rel-latest}-RELEASE-amd64 + +# podman load -i=$OCIBASE/FreeBSD-{rel-latest}-RELEASE-amd64-container-image-toolchain.txz +Getting image source signatures +Copying blob b04531b7d93d done | +Copying config b927c11b2e done | +Writing manifest to image destination +Loaded image: localhost/freebsd-toolchain:{rel-latest}-RELEASE-amd64 +---- + +[[containers-listing]] +=== Listing Images + +[source,shell,subs="attributes"] +---- +# podman images +REPOSITORY TAG IMAGE ID CREATED SIZE +localhost/freebsd-toolchain {rel-latest}-RELEASE-amd64 b927c11b2e8d 13 days ago 660 MB +localhost/freebsd-notoolchain {rel-latest}-RELEASE-amd64 086572f35cc9 13 days ago 152 MB +localhost/freebsd-runtime {rel-latest}-RELEASE-amd64 90c493675429 13 days ago 34.1 MB +localhost/freebsd-dynamic {rel-latest}-RELEASE-amd64 33427d59f990 13 days ago 14.5 MB +localhost/freebsd-static {rel-latest}-RELEASE-amd64 5921c62e76c1 13 days ago 2.92 MB +---- + +Note that the image IDs are reproducible each time, and the `+IMAGE_ID+` column matches the hashes reported when importing the images. + +[[containers-listing-layers]] +=== Listing Layers + +It is possible to show the layers that comprise an image, demonstrating how the `+runtime+` image has three components, each one a layer in its own right: + +[source,shell] +---- +# podman image tree ghcr.io/freebsd/freebsd-runtime:14.2 +Image ID: c5f3e77557a9 +Tags: [ghcr.io/freebsd/freebsd-runtime:14.2] +Size: 35.07MB +Image Layers +├── ID: cd53fb07fb66 Size: 5.449MB Top Layer of: [ghcr.io/freebsd/freebsd-static:14.2] +├── ID: a01d37f7777b Size: 10.4MB Top Layer of: [ghcr.io/freebsd/freebsd-dynamic:14.2] +└── ID: 36b0c80ca1f7 Size: 19.21MB Top Layer of: [ghcr.io/freebsd/freebsd-runtime:14.2] +---- + +[[containers-public-registries]] +=== Using Public Registries + +While the most secure provenance is downloading from https://download.freebsd.org/releases[Official FreeBSD Releases], there are two public container registries managed by the FreeBSD Release and Cluster Admin teams. +The images from both registries are identical, but provide a simpler and more container-friendly workflow. + +* https://hub.docker.com/u/freebsd[Docker Hub] +* https://github.com/orgs/freebsd/packages[GitHub Container Registry] + +[[containers-fetching]] +=== Fetching Containers + +Images can be fetched from a public registry using `+podman pull+`: + +[source,shell,subs="attributes"] +---- +# podman pull ghcr.io/freebsd/freebsd-notoolchain:{rel-latest} +Trying to pull ghcr.io/freebsd/freebsd-notoolchain:{rel-latest}... +Getting image source signatures +Copying blob 711cf0d3c0ef done | +Copying config 086572f35c done | +Writing manifest to image destination +086572f35cc90b1a1d2b7904b63e8a849071b4ba89b15492f6da47a802372aa2 +---- + +[[containers-running]] +=== Running a Container + +Use `+podman run+` to start a container. +The `+-it+` flags allocate an interactive terminal, and `+--rm+` removes the container when it exits. +If no command is specified, the image's default entrypoint is used: + +[source,shell,subs="attributes"] +---- +# podman run -it --rm ghcr.io/freebsd/freebsd-notoolchain:{rel-latest} +# exit +---- + +A specific command can be given as well: + +[source,shell,subs="attributes"] +---- +# podman run -it --rm ghcr.io/freebsd/freebsd-notoolchain:{rel-latest} uname -a +FreeBSD c0038de70651 {rel-latest}-RELEASE FreeBSD {rel-latest}-RELEASE releng/{rel-latest}-n283562-96841ea08dcf GENERIC amd64 +---- + +As the container image does not include the kernel, man:freebsd-version[1] shows different results for the running kernel versus the installed userland. +In this example, a {rel-latest-previous}-RELEASE container runs on a {rel-latest}-RELEASE host: + +[source,shell,subs="attributes"] +---- +# podman run -it --rm ghcr.io/freebsd/freebsd-notoolchain:{rel-latest-previous} /bin/sh +# freebsd-version -ru +{rel-latest}-RELEASE +{rel-latest-previous}-RELEASE +# exit +---- + +[[containers-variations]] +== Useful Variations and Tips + +Almost all of the `+podman run ...+` parameters also apply to `+podman build+`. +Refer to the man:podman-run[1] and man:podman-build[1] manual pages for details. + +Use the `+--rm+` flag to have ephemeral containers clean themselves up afterwards. + +Use `+podman images -a+` to show all downloaded images: + +[source,shell,subs="attributes"] +---- +# podman images -a +REPOSITORY TAG IMAGE ID CREATED SIZE +ghcr.io/freebsd/freebsd-runtime {rel-latest} 90c493675429 2 weeks ago 34.1 MB +---- + +Use `+podman ps -a+` to see running containers: + +[source,shell,subs="attributes"] +---- +# podman ps -a +CONTAINER ID IMAGE COMMAND CREATED STATUS NAMES +3123623bef9b ghcr.io/freebsd/freebsd-runtime:{rel-latest} /bin/sh 4 minutes ago Up 4 minutes admiring_brattain +---- + +[[containers-env-vars]] +=== Environment Variables + +Podman sets a number of default environment variables in each container: + +[source,shell,subs="attributes"] +---- +# podman run -it --rm ghcr.io/freebsd/freebsd-notoolchain:{rel-latest} /usr/bin/env +container=podman +TERM=xterm +PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +HOME=/root +HOSTNAME=60bae3343e6b +---- + +The `+HOSTNAME+` is set to the container ID by default. +It can be overridden with `+--hostname+`, which is useful when the application inside the container needs a stable or meaningful hostname. + +Arbitrary environment variables can be passed in with `+--env+`: + +[source,shell,subs="attributes"] +---- +# podman run --env FOO=BAR -it --rm ghcr.io/freebsd/freebsd-runtime:{rel-latest} +# echo $FOO +BAR +---- + +[[containers-volumes]] +=== Mounting Files and Volumes + +Both files and volumes can be passed into containers using `+--volume+`. +For example, the `+runtime+` image does not contain man:which[1]: + +[source,shell,subs="attributes"] +---- +# podman run -it --rm ghcr.io/freebsd/freebsd-runtime:{rel-latest} +# which +/bin/sh: which: not found +---- + +It can be mounted into the container from the host: + +[source,shell,subs="attributes"] +---- +# podman run -it --rm --volume /usr/bin/which:/usr/bin/which \ + ghcr.io/freebsd/freebsd-runtime:{rel-latest} +# which which +/usr/bin/which +---- + +For a lean image such as `+freebsd-static+`, mount the existing man:pkg-static[8] from the host to bootstrap man:pkg[8] inside the container. +Sharing the host's package cache via `+--volume /var/cache/pkg+` avoids re-downloading packages that have already been fetched, which is particularly useful when building multiple containers. +The `+--no-hosts+` flag prevents mounting `+/etc/hosts+`, which would otherwise interfere with package upgrades. + +[NOTE] +==== +Sharing the package cache only works when the host and container run the same FreeBSD major version, as packages are version-specific. +==== + +[source,shell,subs="attributes"] +---- +# podman run -it --rm \ + --env ASSUME_ALWAYS_YES=true \ + --no-hosts \ + --volume /usr/local/sbin/pkg-static:/bin/pkg-static \ + --volume /var/cache/pkg \ + ghcr.io/freebsd/freebsd-static:{rel-latest} +# pkg-static bootstrap -r FreeBSD +Bootstrapping pkg from pkg+https://pkg.FreeBSD.org/FreeBSD:14:amd64/latest, please wait... +Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done +[fd343e182fa5] Installing pkg-2.1.2... +[fd343e182fa5] Extracting pkg-2.1.2: 100% +---- + +Files and directories from the host can also be mounted into containers: + +[source,shell,subs="attributes"] +---- +# podman run -it --rm \ + --volume /etc/fstab:/etc/fstab \ + --volume /tmp:/var/tmp \ + --volume /tmp \ + ghcr.io/freebsd/freebsd-runtime:{rel-latest} +---- + +[[containers-inspecting]] +=== Inspecting Image Contents + +To list all files in any image, including minimal images with no shell or tools, export the container filesystem and list its contents: + +[source,shell,subs="attributes"] +---- +# podman export $(podman create --rm ghcr.io/freebsd/freebsd-static:{rel-latest} /) | tar tf - +---- + +The container images do not include man:pkg[8] itself, but man:pkg-static[8] can be mounted from the host to query the installed base system packages. +Use `+pkg info -q+` to list package names, or `+pkg info -ql+` to show all files owned by packages: + +[source,shell,subs="attributes"] +---- +# podman run --rm \ + -v /usr/local/sbin/pkg-static:/pkg \ + ghcr.io/freebsd/freebsd-runtime:{rel-latest} /pkg info -q +FreeBSD-audit-lib-{rel-latest} +FreeBSD-bzip2-lib-{rel-latest} +FreeBSD-certctl-{rel-latest} +FreeBSD-clibs-{rel-latest} +FreeBSD-fetch-{rel-latest} +FreeBSD-kerberos-lib-{rel-latest} +FreeBSD-libarchive-{rel-latest} +FreeBSD-libcasper-{rel-latest} +FreeBSD-libexecinfo-{rel-latest} +FreeBSD-libucl-{rel-latest} +FreeBSD-mtree-{rel-latest} +FreeBSD-ncurses-lib-{rel-latest} +FreeBSD-openssl-{rel-latest} +FreeBSD-openssl-lib-{rel-latest} +FreeBSD-pam-lib-{rel-latest} +FreeBSD-pkg-bootstrap-{rel-latest} +FreeBSD-rc-{rel-latest} +FreeBSD-runtime-{rel-latest} +FreeBSD-xz-lib-{rel-latest} +FreeBSD-zlib-{rel-latest} +FreeBSD-zoneinfo-{rel-latest} +FreeBSD-zstd-lib-{rel-latest} +---- + +When the host and container run different FreeBSD major versions, man:pkg[8] detects the ABI by inspecting `+/bin/sh+` inside the container, and will warn about the mismatch: + +[source,shell] +---- +# podman run -it --rm \ + --env ASSUME_ALWAYS_YES=true \ + --no-hosts \ + --volume /usr/local/sbin/pkg-static:/usr/sbin/pkg \ + --volume /var/cache/pkg \ + ghcr.io/freebsd/freebsd-runtime:14.4 /bin/sh +# pkg bootstrap +pkg: Warning: Major OS version upgrade detected. Running "pkg bootstrap -f" recommended +---- + +For images without `+/bin/sh+`, such as `+freebsd-dynamic+`, man:pkg-static[8] cannot detect the system ABI at all. + +[[containers-networking]] +== Networking and Name Resolution + +By default, the Podman suite of tools will use man:mount_nullfs[8] to mount `+/etc/resolv.conf+` and a modified `+/etc/hosts+` from the jail host, as well as a `+.containerenv+` file in `+/var/run+`. + +This hosts file will conflict with base system package upgrades, so it is advised to skip this during `+podman build+` runs, but leave it enabled during production deployment: + +[source,shell] +---- +# podman run --dns 1.2.3.4 ... +# podman build --no-hosts ... +---- + +The `+--dns+` flag specifies a custom DNS server instead of mounting `+/etc/resolv.conf+` from the host. +The `+--no-hosts+` flag prevents mounting `+/etc/hosts+` from the host. + +[[containers-building]] +== Building Custom Images + +Custom images can be built directly from tarballs (for example those from man:poudriere[8], or official release tarballs), from existing tagged images, or bootstrapped from man:pkg[8]. + +[[containers-building-base]] +=== Building from Tarballs + +A custom image can be made from any FreeBSD root filesystem tarball using man:podman-import[1]. +The tarball can be an official release `+base.txz+`, one produced by man:poudriere-image[8], or your own custom tarball. +The final tagged image can be pushed to registries, modified, and used for further image creation. + +[source,shell,subs="attributes"] +---- +# podman import --os freebsd \ + --arch amd64 \ + --message 'Import FreeBSD {rel-latest}-RELEASE base.txz' \ + https://download.freebsd.org/releases/amd64/{rel-latest}-RELEASE/base.txz +Downloading from "https://download.freebsd.org/releases/amd64/{rel-latest}-RELEASE/base.txz" +Getting image source signatures +Copying blob 3768988b151c done | +Copying config 332bec83a4 done | +Writing manifest to image destination +sha256:332bec83a428efa6c9f7d1f8b3fbd589c5613d938dca7da5125240c89f4f3dcd + +# podman image tag 332bec83a428 localhost/freebsd-base:{rel-latest} + +# podman image ls +REPOSITORY TAG IMAGE ID CREATED SIZE +localhost/freebsd-base {rel-latest} 332bec83a428 5 minutes ago 715 MB +---- + +[[containers-building-containerfile]] +=== Building from Containerfiles + +Images can be built using a `+Containerfile+`, similar to a Dockerfile. +In most cases, the `+freebsd-notoolchain+` image is the most convenient base, as it includes nearly all standard system tools. +For leaner containers, start from `+freebsd-static+`, `+freebsd-dynamic+`, or `+freebsd-runtime+` as appropriate, and add only what is needed: + +[source,dockerfile,subs="attributes"] +---- +# Containerfile +FROM ghcr.io/freebsd/freebsd-notoolchain:{rel-latest} +RUN env ASSUME_ALWAYS_YES=yes IGNORE_OS_VERSION=yes pkg bootstrap -r FreeBSD +RUN pkg update +RUN pkg upgrade -y +RUN pkg clean -y +ENTRYPOINT /bin/sh +---- + +Build the image with man:podman-build[1]: + +[source,shell,subs="attributes"] +---- +# podman build --no-hosts --squash \ + -t localhost/freebsd-base:{rel-latest} \ + -f ./Containerfile +---- + +Further images can be layered on top: + +[source,dockerfile,subs="attributes"] +---- +# Containerfile.webserver +FROM localhost/freebsd-base:{rel-latest} +RUN pkg install -y lang/python3 +RUN pkg clean -y +ENTRYPOINT /usr/local/bin/python3 -m http.server +---- + +[source,shell,subs="attributes"] +---- +# podman build --no-hosts \ + -t localhost/python-www:{rel-latest} \ + -f ./Containerfile.webserver +---- + +Run the web server in the background with `+--detach+`, and use `+-p+` to map a host port to the container port. +In this example, the container listens on its default port 8000, which is mapped to port 8888 on the host: + +[source,shell,subs="attributes"] +---- +# c=$(podman run --detach --rm -p 8888:8000 localhost/python-www:{rel-latest}) +---- + +The port mapping uses man:pf[4] redirect rules. +These rules redirect traffic arriving over the network, so the mapped port must be accessed from another host, or via the host's network-facing IP address: + +[source,shell] +---- +$ fetch -v http://172.16.2.10:8888/ +resolving server address: 172.16.2.10:8888 +requesting http://172.16.2.10:8888/ +remote size / mtime: 784 / 0 +fetch.out 784 B 3791 kBps 00s +---- + +To access the container directly from the host, use the container's own IP on its listening port: + +[source,shell] +---- +# podman inspect -f '{{.NetworkSettings.IPAddress}}' $c +10.88.0.91 +# fetch -v http://10.88.0.91:8000/ +resolving server address: 10.88.0.91:8000 +requesting http://10.88.0.91:8000/ +remote size / mtime: 784 / 0 +fetch.out 784 B 25 MBps 00s +---- + +[NOTE] +==== +Port mappings via `+-p+` are handled by man:pf[4] redirect rules and only apply to traffic arriving over the network. +Traffic originating from the jail host itself to `+localhost+` or `+127.0.0.1+` will not be redirected. +From the host, use the container's IP address directly instead. +==== + +[[containers-advanced]] +== Advanced Usage + +[[containers-daemonising]] +=== Daemonising Podman + +FreeBSD's Podman implementation provides two man:rc[8] services: + +* The `+podman+` service runs at boot and ensures that all containers marked with `+restart-policy=always+` are started. +* The `+podman_service+` service runs the https://docs.podman.io/en/latest/_static/api.html[Podman API service], providing a REST API and listening on `+/var/run/podman/podman.sock+` by default. + +Enable them as follows: + +[source,shell] +---- +# sysrc podman_enable=YES +# sysrc podman_service_enable=YES +# sysrc podman_service_flags='--time 0' +# service podman start +# service podman_service start +---- + +Containers that should restart automatically after a host reboot must be run with the `+--restart=always+` option: + +[source,shell] +---- +# podman run --detach --restart=always my-image +---- + +[[containers-linux]] +=== Running Linux Containers + +It is possible to run many Linux container images using FreeBSD's Linux emulation: + +[source,shell] +---- +# service linux onestart +# podman run --rm --os=linux docker.io/alpine cat /etc/os-release | head -1 +Trying to pull docker.io/library/alpine:latest... +Getting image source signatures +Copying blob 4abcf2066143 done | +Copying config 05455a0888 done | +Writing manifest to image destination +NAME="Alpine Linux" +---- + +[NOTE] +==== +Linux containers that depend on systemd, expect to be PID 1, or rely on Linux-specific kernel interfaces may not work correctly under FreeBSD's Linux emulation. +==== + +[[containers-local-registry]] +=== Running a Local Registry + +A registry is a web server for storing and sharing OCI images. +The https://freshports.org/sysutils/docker-registry[sysutils/docker-registry] port provides a simple registry suitable for local development or a small organisation. +https://zotregistry.dev/[Zot Registry] is a more comprehensive modern alternative. + +[source,shell] +---- +# pkg install -r FreeBSD sysutils/docker-registry +# zfs create zroot/var/db/registry +# chown -R www:www /var/db/registry +# sysrc docker_registry_enable=YES +# sysrc docker_registry_user=www +# sed -i '' -E -e 's,rootdirectory:.+,rootdirectory: /var/db/registry,' \ + /usr/local/etc/docker-registry/config.yml +# touch /usr/local/etc/docker-registry/htpasswd +# chown -R root:www /usr/local/etc/docker-registry/config.yml \ + /usr/local/etc/docker-registry/htpasswd +# chmod 0640 /usr/local/etc/docker-registry/config.yml \ + /usr/local/etc/docker-registry/htpasswd +---- + +A sample configuration for `+/usr/local/etc/docker-registry/config.yml+`: + +[source,yaml] +---- +version: 0.1 +log: + fields: + service: registry +storage: + cache: + blobdescriptor: inmemory + filesystem: + rootdirectory: /var/db/registry +http: + addr: :5000 + secret: <random-secret> + headers: + X-Content-Type-Options: [nosniff] +auth: + htpasswd: + realm: basic-realm + path: /usr/local/etc/docker-registry/htpasswd +health: + storagedriver: + enabled: true + interval: 10s + threshold: 3 +---- + +Start the registry and verify it is running: + +[source,shell] +---- +# service registry start +# curl -u admin:passwd http://localhost:5000/v2/ +---- + +[TIP] +==== +It is recommended to serve the registry over TLS, behind a reverse proxy such as nginx or haproxy. +==== + +[[containers-syncing]] +=== Synchronising Registries + +It is possible to synchronise official FreeBSD images into a local registry using `+skopeo+`. +The following example replicates all FreeBSD-related images from https://hub.docker.com/u/freebsd[Docker Hub] to a local registry: + +[source,yaml] +---- +# sync.yml +docker.io: + images: + freebsd/freebsd-static: [] + freebsd/freebsd-dynamic: [] + freebsd/freebsd-runtime: [] +---- + +[source,shell] +---- +$ skopeo sync --src yaml --dest docker --all sync.yml cr.example.net/freebsd +INFO[0000] Processing repo repo=freebsd/freebsd-static +INFO[0000] Querying registry for image tags repo=freebsd/freebsd-static +... +INFO[0771] Synced 42 images from 3 sources +---- + +So long as the credentials in `+/root/.config/containers/auth.json+` are up to date, root access is not required to sync images. + +[[containers-registries-auth]] +=== Working with Container Registries + +Registries generally require authentication for push operations, and optionally for pull. +Credentials are saved in `+/root/.config/containers/auth.json+` by default after login: + +[source,shell] +---- +# podman login ghcr.io -u username +Password: +Login Succeeded! +---- + +Images can then be pushed to a registry: + +[source,shell,subs="attributes"] +---- +# podman push localhost/freebsd-runtime:{rel-latest} \ + docker://ghcr.io/example/freebsd-runtime:{rel-latest} +Getting image source signatures +Copying blob bbacfdfbe140 done | +Copying config 786f2592a8 done | +Writing manifest to image destination +---- + +Public images can be pulled without authentication: + +[source,shell,subs="attributes"] +---- +# podman pull ghcr.io/freebsd/freebsd-runtime:{rel-latest} +---- + +[[containers-building-sources]] +== Building Container Images from Sources + +The FreeBSD release tooling provides an `+oci-release+` target to build the necessary images for `+podman load+` to use. +It requires building world, kernel, and base system packages first, so this can take a while. + +The following example builds `+arm64+` images: + +[source,shell] +---- +# cd /usr/src \ + && make TARGET_ARCH=aarch64 TARGET=arm64 -s -j32 buildworld \ + && make TARGET_ARCH=aarch64 TARGET=arm64 KERNCONF=GENERIC -s -j32 buildkernel \ + && make TARGET_ARCH=aarch64 TARGET=arm64 KERNCONF=GENERIC -s -j32 packages \ + && cd ./release \ + && make TARGET_ARCH=aarch64 TARGET=arm64 -j32 -s clean \ + && make KERNCONF=GENERIC TARGET_ARCH=aarch64 TARGET=arm64 WITH_OCIIMAGES=yes \ + -s oci-release +---- + +And similarly for `+amd64+`: + +[source,shell] +---- +# cd /usr/src \ + && make TARGET_ARCH=amd64 TARGET=amd64 -s -j32 buildworld \ + && make TARGET_ARCH=amd64 TARGET=amd64 KERNCONF=GENERIC -s -j32 buildkernel \ + && make TARGET_ARCH=amd64 TARGET=amd64 KERNCONF=GENERIC -s -j32 packages \ + && cd ./release \ + && make TARGET_ARCH=amd64 TARGET=amd64 -s -j32 clean \ + && make KERNCONF=GENERIC TARGET_ARCH=amd64 TARGET=amd64 \ + WITH_OCIIMAGES=yes -s oci-release +---- + +[[containers-cross-arch]] +== Cross-Architecture Testing + +The `+emulators/qemu-user-static+` package allows testing container images built for other architectures. +After installing the package, enable and start the service to register image activators for all emulated architectures: + +[source,shell] +---- +# pkg install emulators/qemu-user-static +# service qemu_user_static enable +# service qemu_user_static start +---- + +Then run an `+arm64+` image on an `+amd64+` host by mounting the appropriate QEMU binary: + +[source,shell] +---- +# podman run -it --rm --arch=arm64 \ + -v /usr/local/bin/qemu-aarch64-static:/usr/local/bin/qemu-aarch64-static \ + ghcr.io/freebsd/freebsd-runtime:14.snap uname -a +FreeBSD 73d4049d10b2 14.2-RELEASE-p1 FreeBSD 14.2-RELEASE-p1 GENERIC arm64 +---- + +[[containers-troubleshooting]] +== Troubleshooting + +[[containers-troubleshooting-hosts]] +=== Package Upgrades and Mounted Files + +Podman mounts `+/etc/resolv.conf+` and `+/etc/hosts+` into containers using man:mount_nullfs[8]. +These mounts can cause errors during man:pkg[8] operations with messages such as `+Cross-device link+`: + +[source] +---- +pkg: Fail to rename /etc/.pkgtemp.hosts.VeIDrHM0sKN5 -> /etc/hosts:Cross-device link +---- + +To avoid this, use `+--no-hosts+` and `+--dns+` flags: + +[source,shell,subs="attributes"] +---- +# podman run --no-hosts --dns 1.1.1.1 -it --rm \ + ghcr.io/freebsd/freebsd-runtime:{rel-latest} /bin/sh +---- + +[[containers-troubleshooting-networking]] +=== Checking Networking + +Start with the firewall rules from `+/usr/local/etc/containers/pf.conf.sample+`. +Test connectivity step by step from within a container: + +[source,shell,subs="attributes"] +---- +# podman run --no-hosts --dns 1.1.1.1 \ + --env ASSUME_ALWAYS_YES=1 \ + -it --rm \ + ghcr.io/freebsd/freebsd-runtime:{rel-latest} \ + /bin/sh + +# route show 1.1.1.1 + route to: 1.1.1.1 +destination: default + gateway: 10.88.0.1 + interface: eth0 + +# ping -c 3 10.88.0.1 +PING 10.88.0.1 (10.88.0.1): 56 data bytes +64 bytes from 10.88.0.1: icmp_seq=0 ttl=64 time=0.083 ms +... + +# ping -c 3 1.1.1.1 +PING 1.1.1.1 (1.1.1.1): 56 data bytes +64 bytes from 1.1.1.1: icmp_seq=0 ttl=60 time=0.920 ms +... + +# fetch -v http://1.1.1.1/ +... + +# pkg -dd bootstrap -r FreeBSD +---- + +If man:ping[8] fails, check the firewall rules for blocked ICMP. +If man:fetch[1] fails, check the firewall rules for blocked TCP on ports 80 and 443. +If man:pkg[8] fails but ICMP and TCP work, this may be a fragmentation or MTU problem. +Add `+set reassemble yes no-df+` to `+/etc/pf.conf+` and restart the firewall. + +For debugging blocked packets, enable `+log+` on all block rules in `+/etc/pf.conf+` and use man:tcpdump[1] on the man:pflog[4] device: + +[source,shell] +---- +# service pflog onerestart +# tcpdump -vvveni pflog0 icmp or net 10.88.0.0/16 +---- + +[[containers-troubleshooting-mtu]] +=== MTU Consistency + +Ensure MTU values are consistent across the primary network interface, the container bridge, and the container itself. +Mismatched MTU values can cause packet fragmentation and connectivity failures. + +In the following example, `+vtnet0+` has MTU 9000, but the bridge and containers have MTU 1500. +In general, align the internal layout to match the external network: + +[source,shell] +---- +# ifconfig vtnet0 mtu 9000 +# ifconfig cni-podman0 mtu 9000 +---- + +Inside the container: + +[source,shell] +---- +# ifconfig eth0 mtu 9000 +---- + +The bridges and epairs used by Podman are ephemeral and will need to be configured for each bridge creation and each container. +In some cloud environments, adding `+set reassemble yes no-df+` to `+/etc/pf.conf+` may also be required.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6a52ac74.3646c.35322aa5>