From nobody Tue Apr 28 19:14:27 2026 X-Original-To: dev-commits-ports-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4g4qql54zKz6by9W for ; Tue, 28 Apr 2026 19:14:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4g4qql2VRFz443Z for ; Tue, 28 Apr 2026 19:14:27 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1777403667; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=U1+Bu20yk7pegJwflN3DDxZAAZREZc8LHKLGUhHVH10=; b=EB+4ECE9ku0kYu0elt37SdqHROB5JJ42PRIO72e72ACDND6Leh+QAJgD1fBqRAGDetRo2G 9TQV31oUPkrbRxtV4XMMrh/fUvlzkuSELy6SHzspLCkfGCRyw/A+3R7sS5GeswMz80bjNP Yw5iD7yvLW96/Q17eCms+iMMy8JnKCVxrs7DiFB7KbcE8/HGohqXSCqvF+mitqCnkQd6dM OA0cKKsdd7ScmdY/yjEbL4ynF71ozjrK3/ThwdZPVKJ7w6mg7SVxfIiP0zed/6uUfowTIr hrX6woHbgpKsDWw9xjgyC7yj7MK5m56hs4/6KpUFgGp/CjUjlgBjq1Y2fgkdNA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1777403667; a=rsa-sha256; cv=none; b=QoOTfnUfJHsItW6EG8J5kaGSGgtA6hkpFECHcW8nBoFQhjG8QoAfRzUyoJDD8HPKtYDj68 UxOKTioUABFNWyrJ2lwFrr/wJW0KI612vYtwH0o7jvC3d/8bxfUjS+up57z7dTkUdbwhF9 yr1O0e926qjMiKhVgtfdbvq8p8WIshRNMR3887kTmsTgqD7VRKUo8ZUWpIznqTBjk/tP2h ewgWh+awW3cpaRRz90vwNmIFCcEo5McEO9QS55mmTWHy4S3Qhk6JuXbIM4QXhb0oXkQqAp UZfYiT6A8FVMU+8G7cx29O/z4FTlCx01V12UdQ+0XdkFooIY+EY6jN0ggT+lqg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1777403667; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=U1+Bu20yk7pegJwflN3DDxZAAZREZc8LHKLGUhHVH10=; b=CFyFoHEIbbgvCE4lL6fTd9LWAj3lUyujtNpR8T9yoDi2ZpePIl259TkWh8THuihjMJ44TQ 6TAr8btdWtkZJr2zSedT1A+0lAR1o5W68cR7WjAXayVLhFR1/sxAQW4mm6KWa2WO38Sep9 srrEnsjoA064+stE2XmSkqu0d1dW3jxq+YLh60anZt0Bw7v3W9LSx0/I+lE0YHsgwH6aIA 44RBG9p4T/fCa2Pj/Mfh6tXziiBqB0oO0lnob4TVg1KgRECC7LfXdFaFU+gQT3mzkpw16w 8/rXCgOP/ax9Fb11Qj0wd5jS1YbfzQMFVyQTJKXgqNNBfBFTsFnh6mAKYbLb3g== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4g4qql1qMsz16Dj for ; Tue, 28 Apr 2026 19:14:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 2677e by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 28 Apr 2026 19:14:27 +0000 To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Yuri Victorovich Subject: git: d31fe5950007 - main - misc/llama-cpp: Multiple changes List-Id: Commits to the main branch of the FreeBSD ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-ports-main@freebsd.org Sender: owner-dev-commits-ports-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: yuri X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: d31fe59500079cb7fe4879ef6f19d313024474ee Auto-Submitted: auto-generated Date: Tue, 28 Apr 2026 19:14:27 +0000 Message-Id: <69f10713.2677e.77087f08@gitrepo.freebsd.org> The branch main has been updated by yuri: URL: https://cgit.FreeBSD.org/ports/commit/?id=d31fe59500079cb7fe4879ef6f19d313024474ee commit d31fe59500079cb7fe4879ef6f19d313024474ee Author: Yuri Victorovich AuthorDate: 2026-04-28 19:11:06 +0000 Commit: Yuri Victorovich CommitDate: 2026-04-28 19:14:22 +0000 misc/llama-cpp: Multiple changes 1. Allow for multiple models to be selected at the run-time 2. Remove the leftover VULKAN option. VULKAN is enabled in misc/ggml. PR: 294795 (allow multiple models at run-time) Requested by: Ivan Rozhuk --- misc/llama-cpp/Makefile | 10 +++------- misc/llama-cpp/files/llama-server.in | 17 ++++++++--------- misc/llama-cpp/pkg-message | 9 +++++++++ 3 files changed, 20 insertions(+), 16 deletions(-) diff --git a/misc/llama-cpp/Makefile b/misc/llama-cpp/Makefile index cde8ceb39f81..d56bdbf3dd40 100644 --- a/misc/llama-cpp/Makefile +++ b/misc/llama-cpp/Makefile @@ -1,6 +1,7 @@ PORTNAME= llama-cpp DISTVERSIONPREFIX= b DISTVERSION= 8895 +PORTREVISION= 1 CATEGORIES= misc # machine-learning MAINTAINER= yuri@FreeBSD.org @@ -36,8 +37,8 @@ CMAKE_TESTING_ON= LLAMA_BUILD_TESTS USER= nobody SUB_LIST= USER=${USER} -OPTIONS_DEFINE= CURL EXAMPLES VULKAN -OPTIONS_DEFAULT= CURL VULKAN +OPTIONS_DEFINE= CURL EXAMPLES +OPTIONS_DEFAULT= CURL EXAMPLES OPTIONS_SUB= yes CURL_DESCR= Use libcurl to download model from an URL @@ -47,11 +48,6 @@ CURL_LIB_DEPENDS= libcurl.so:ftp/curl EXAMPLES_CMAKE_BOOL= LLAMA_BUILD_EXAMPLES -VULKAN_CMAKE_BOOL= GGML_VULKAN -VULKAN_BUILD_DEPENDS= glslc:graphics/shaderc \ - vulkan-headers>0:graphics/vulkan-headers -VULKAN_LIB_DEPENDS= libvulkan.so:graphics/vulkan-loader - BINARY_ALIAS= git=false \ python=${PYTHON_CMD} # for tests diff --git a/misc/llama-cpp/files/llama-server.in b/misc/llama-cpp/files/llama-server.in index 6eec15953978..82992f661ca3 100644 --- a/misc/llama-cpp/files/llama-server.in +++ b/misc/llama-cpp/files/llama-server.in @@ -13,7 +13,7 @@ # llama_server_user (str): llama_server daemon user # Default: %%USER%% # llama_server_model (str): AI model that llama-server will use -# Default: "" (required) +# Default: "" (not required) # llama_server_args (str): Additional arguments for llama-server # Default: "" (optional) # llama_server_log (str): Log file that llama-server will write log to @@ -38,20 +38,19 @@ run_command="%%PREFIX%%/bin/llama-server" procname="${run_command}" pidfile=${llama_server_pidfile} command=/usr/sbin/daemon -command_args="-f -t ${name} -p ${pidfile} -o ${llama_server_log} ${run_command} -m ${llama_server_model} ${llama_server_args}" +command_args="-f -t ${name} -p ${pidfile} -o ${llama_server_log} ${run_command} ${llama_server_args}" start_precmd="llama_server_precmd" llama_server_chdir=/tmp llama_server_precmd() { # check model - if [ -z "${llama_server_model}" ]; then - echo "llama_server_model isn't set, it is required" - exit 1 - fi - if [ ! -f "${llama_server_model}" ]; then - echo "llama_server_model isn't a file" - exit 1 + if [ -n "${llama_server_model}" ]; then + if [ ! -f "${llama_server_model}" ]; then + echo "llama_server_model isn't a file" + exit 1 + fi + command_args="${command_args} -m ${llama_server_model}" fi # ensure that the log file exists and has right permissions diff --git a/misc/llama-cpp/pkg-message b/misc/llama-cpp/pkg-message index 157a4db6ea78..15534c8fd534 100644 --- a/misc/llama-cpp/pkg-message +++ b/misc/llama-cpp/pkg-message @@ -22,6 +22,15 @@ and navigate to http://localhost:8080: > llama_server_model=/path/to/models/llama-2-7b-chat.Q4_K_M.gguf > llama_server_args="--device Vulkan0 -ngl 27" +In order to use the multi-model feature do not use llama_server_model. +Instead add the argument "--models-preset /path/to/models.ini" +Add pre-downloaded models into models.ini, for example: +[Qwen3.5-35B-A3B-Uncensored] +model = /path/to/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf + +You can switch to the CPU-only operation by choosing the port option +VULKAN=OFF in misc/ggml (not in llama-cpp). + EOM } ]