From owner-freebsd-virtualization@FreeBSD.ORG Mon May 5 18:14:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A698129; Mon, 5 May 2014 18:14:02 +0000 (UTC) Received: from mail-qg0-x22b.google.com (mail-qg0-x22b.google.com [IPv6:2607:f8b0:400d:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E0135CFF; Mon, 5 May 2014 18:14:01 +0000 (UTC) Received: by mail-qg0-f43.google.com with SMTP id 63so2291131qgz.2 for ; Mon, 05 May 2014 11:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=QyuEsIr7mVZdPViRGLygGC9rJGUFzPZksDcO3DXPCoU=; b=FRnv9poQxZn2O6UDRB3cSe+e4deZm2rXVDjPKS9fkD4pLKDjgBQAV0ABC68FeUU1IM Jgu/2SKTG7Ohj4h42c4mjHY89pFhC/3azECZRlgy44Oa0pH/usKdGu1mtxjptp/t8ykG Y4USrvFn3qWLG3uvDYAQ2Dl0d2hLBjijG2Zx8gXypVqTd3fPO6yXyYLS78cq47RnkAkv dg/eqAS8amFvPJLMhjbKCiTZEBBhZnseCSa5u/kZZQji7e3fm5F8scLTlUrEfdUPHpyR wRNmr1BkRYi9l48qejJBvSkjIDAm3HefuEzGpeUa4LFJHYgQzbe8S8ZtAChB6qVaRIka ydhg== MIME-Version: 1.0 X-Received: by 10.224.72.12 with SMTP id k12mr48232900qaj.81.1399313640659; Mon, 05 May 2014 11:14:00 -0700 (PDT) Received: by 10.140.50.235 with HTTP; Mon, 5 May 2014 11:14:00 -0700 (PDT) In-Reply-To: <20140501143005.GA91029@kloomba> References: <20140427104511.GA7804@kloomba> <20140501143005.GA91029@kloomba> Date: Mon, 5 May 2014 11:14:00 -0700 Message-ID: Subject: Re: [PATCH] Flexible vcpu pinning configuration From: Neel Natu To: Roman Bogorodskiy Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 18:14:02 -0000 Hi Roman, On Thu, May 1, 2014 at 7:30 AM, Roman Bogorodskiy wrote= : > Neel Natu wrote: > >> Hi Roman, >> >> On Sun, Apr 27, 2014 at 3:45 AM, Roman Bogorodskiy w= rote: >> > I've created an initial version of the patch which allows more flexibl= e >> > vcpu pinning configuration. >> > >> > Current schema is: >> > >> > bhyve -p N >> > >> > pins vcpu i to hostcpu N + i. >> > >> > The propsed extension is: >> > >> > bhyve -p N:M .... -p 0:1 -p 3:5 >> > >> > which pins vcpu N to host pcpu M. Option needs to be specified >> > individually for each vcpu. >> > >> > So it works like that for me: >> > >> > sudo /usr/sbin/bhyve -p 0:0 -p 1:3 -c 2 ... >> > >> > # sudo cpuset -g -t 100262 >> > tid 100262 mask: 0 >> > # sudo cpuset -g -t 100264 >> > tid 100264 mask: 3 >> > >> > PS I used cpumat_t* array to store these values instead of int, becaus= e >> > if the idea is OK, I'll extend it to support ranges like e.g. cpuset(1= ) >> > supports, e.g.: "1:2-5". >> > >> > The questions are: >> > >> > - Is it OK to chance '-p' arg syntax or it's better to introduce a ne= w >> > one? >> > >> >> I think we can reuse the "-p" option unless anybody objects vociferously= . >> >> > - Is the syntax OK (currently: 'vcpu:pcpu', later >> > 'vcpu:pcpuN-pcpuM,pcpuX")? >> >> Yup, I think that works fine. >> >> The patch looks good in general but I have a few comments: >> >> - Scope of 'vcpupmap[]' should be restricted to 'static'. >> >> - usage() and man page need to be updated. >> >> - pincpu_parse(): >> The option parsing can be made much easier by using: >> >> if (sscanf(str, "%d:%d", &vcpu, &pcpu) =3D=3D 2) { >> /* success */ >> } else { >> return (-1); >> } >> >> If the same vcpu is specified multiple times then we should >> malloc(sizeof(cpuset_t)) only the first time: >> >> if (vcpumap[vcpu] !=3D NULL) >> mask =3D vcpumap[vcpu]; >> else >> mask =3D malloc(sizeof(cpuset_t)); >> >> We need to range-check 'vcpu' before using it as an index into the >> 'vcpumap[]' array. >> >> best >> Neel > > Attached an updated patch. A slightly modified version was submitted: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D265376 > > I'm still inclined to use something like parselist() from > usr.bin/cpuset/cpuset.c, but I don't want to copy/paste and I don't know > where it'd make sense to move it so it was usable outside of cpuset? > Hmm, not sure really =E2=80=A6 but you can get started by copying it into bhyve and if there is a better place to put it we can do that too. > PS While reading bhyverun.c, I think I spotted a typo: in fbsdrun_deletec= pu() > error message says fprintf(stderr, "addcpu: .... Should it be "deletecpu:= " instead? Thanks. This is fixed now: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D265366 best Neel > > Roman Bogorodskiy