From owner-soc-status@freebsd.org Sun Jul 12 16:33:31 2015 Return-Path: Delivered-To: soc-status@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9314899B4D7 for ; Sun, 12 Jul 2015 16:33:31 +0000 (UTC) (envelope-from prasadjoshi.linux@gmail.com) Received: from mail-vn0-x233.google.com (mail-vn0-x233.google.com [IPv6:2607:f8b0:400c:c0f::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C7601A4C for ; Sun, 12 Jul 2015 16:33:31 +0000 (UTC) (envelope-from prasadjoshi.linux@gmail.com) Received: by vnbf190 with SMTP id f190so26460819vnb.6 for ; Sun, 12 Jul 2015 09:33:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=aG0xZ+x0Eu6wEti1LBC6p5NRA18SUxwDEmIPDpDh8vo=; b=YhG5AZPDMi2NLz0M7xnRFlEWfZm5CB4TdM/EU4zJGwlw/gNPcWJ9EkyDYtK56pALCH 3uBjP1+x5lqaDJm8/jVv/Elo8qGHP4shC1BfrBPMr1YTL91maSvEWsaMxw6SPgeeBfAl joPLDDbCsi8QXTvKI42QnDUIV+ekQ8gjOmtjAGm3AJa9OHDOOwkMq7zgBiFyduhcUC9j zdeCr286vg+vgQmil9s7BoSOzKuRRKmzWMsVvi0h47m6CnszHOiJ/hu+fUJVKW/k3gch CNOGfscS+IbQzoPDxPuIT59xD2x54vUla2duspaNG/dWrhrMwMvTfu2R0ixXwMGqwAH0 xLZw== MIME-Version: 1.0 X-Received: by 10.52.142.228 with SMTP id rz4mr27246360vdb.61.1436718809660; Sun, 12 Jul 2015 09:33:29 -0700 (PDT) Received: by 10.31.50.6 with HTTP; Sun, 12 Jul 2015 09:33:29 -0700 (PDT) In-Reply-To: References: Date: Sun, 12 Jul 2015 22:03:29 +0530 Message-ID: Subject: Re: [gsoc15] dynamically discover bes From: Prasad Joshi To: soc-status@freebsd.org Cc: Xin LI Content-Type: text/plain; charset=UTF-8 X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jul 2015 16:33:31 -0000 Status report for past two weeks ========================= - Last two weeks I struggled to get non-active BE to boot completely - after zfs mount -a all files in /dev/ used to disappear. Finally disabling canmount property on each dataset helped me boot non-active BEs. - I added code to pass mountpoint information from gptzfsboot to zfsloader, then from zfsloader I could set vfs.root.mountfrom env variable. - Added code to find timestamp of each BE clone, with this change BEs could now be sorted on either timestamp, BE name, or object number Tasks For coming week ================== 1. start the initial implementation of menu functionality. On Tue, Jun 30, 2015 at 7:26 PM, Prasad Joshi wrote: > Week 5 Update > > - I did not work on GSoC for 3 days in this week. Had to attend full > day sessions in the University. > - Last week I was faced with a problem with booting alternate > (non-active) BE. Plan in this week was to locate the problem. It seems > like, during bootup zfsloader needs to initialize libzfs library. The > libzfs library initialization fails because '/dev/zfs' cannot be > opened. I verified the zfs kernel modules, which create the device are > loaded. I think, and as suggested by mentor, I will have to set > vfs.root.mountfrom environment variable during bootup. > > Thanks and Regards, > Prasad > > On Mon, Jun 22, 2015 at 11:42 PM, Prasad Joshi > wrote: >> Week 4 update >> >> - I have been able to discover BEs on console. I could detect active BE. >> Created list of BEs. Code to sort BEs on object number, name, or timestamp >> is added. >> >> - I am able to boot from nonactive BE to some extent. At the moment, code >> requires me to enter BE number to boot from. >> >> Tasks next week >> 1. Identify a problem with be bootup. >> 2. Pass mount point info through env variable to loader. >> 3. Start with console based menu. >> >> Thanks and Regards, >> Prasad >> >> On Jun 15, 2015 7:48 PM, "Prasad Joshi" wrote: >>> >>> Week 3 status >>> ========== >>> After understanding on disk representation of snapshots and clones, I >>> have been able to find names of the BEs created using beadm command. I >>> could print the BE names on console. >>> >>> The next task would be to convert BE names to object numbers, create >>> list of BEs. >>> >>> Thanks and Regards, >>> Prasad >>> >>> On Tue, Jun 9, 2015 at 6:36 AM, Prasad Joshi >>> wrote: >>> > Last week I mostly worked on understanding beadm and gptzfsboot code >>> > >>> > beadm create prepares new BE by creating a snapshot and clone of that >>> > snapshot. beadm activate command sets bootfs property of the POOL. >>> > bootfs property contains object number of active dataset object. >>> > During bootup gptzfsboot probes all the disks, creating SPA for any >>> > valid pool. gptzfsboot assumes the first pool it finds as a primary >>> > pool, it then reads meta object set, then tries to find object nunber >>> > of active dataset object either through >>> > a. bootfs - it would be set if BE was already created >>> > b. mos->properties_zap->root_dataset->dd_head_dataset_obj (through >>> > root_datasets bonus buffer) >>> > Once the object number is obtaind gptzfsboot mounts the dataset. >>> > >>> > After mounting, few files are looked up like /boot/config or >>> > /boot.config for presense of boot command (did not went into details >>> > of this). Then (if keyboard is not hit), gptzfsboot execs >>> > /boot/zfsloader. If gptzfsboot is interrupted by keyboard, then it >>> > displays default BE or POOL it is trying to boot from (using >>> > zfs_rlookup() to map active dataset object to string BE name). Though >>> > I haven't checked this but through serial console user would be >>> > allowed to enter other pool or BE to boot from. >>> > >>> > I could not go into details of how gptzfsboot reads /boot/zfsloader from >>> > disk. >>> > >>> > Pending Tasks >>> > ============= >>> > 1. Understand upon keyboard interruption, how user entered zfs paths >>> > (format [zfs:pool/filesystem:][/path/to/loader]) are converted in >>> > object numbers? >>> > 2. Learn ZFS on disk format in more details so as to identify active >>> > dataset object numbers of all BEs. Once the object numbers are >>> > available, I can use zfs_rlookup() function to map object number to >>> > printable pool name. >>> > 3. Prepare library for console based menu. >>> > >>> > Plan for next week >>> > ================== >>> > Pending task 2 above From owner-soc-status@freebsd.org Mon Jul 13 20:34:35 2015 Return-Path: Delivered-To: soc-status@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A73E299C9E0 for ; Mon, 13 Jul 2015 20:34:35 +0000 (UTC) (envelope-from mihai.carabas@gmail.com) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E97B10A8 for ; Mon, 13 Jul 2015 20:34:35 +0000 (UTC) (envelope-from mihai.carabas@gmail.com) Received: by wiga1 with SMTP id a1so80413722wig.0 for ; Mon, 13 Jul 2015 13:34:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=96beqTTNRuc33Jw/zcCEnkusU3G2yFCzL4/JcEZJ6eE=; b=Vfy0tSOJReNhYRMA0bcnOR1zR3Ij4bQCk+gUGmsjnaA4CjOn8r4HQa9FgdBStxdBbr uFND7wHspEgpKc2PLzeXZHy9NWGxpHR7IcQbKa4uGIat5LIOeahETpZXemK4RugkChwJ SSLn37jSb4s/KKPz9jHLJwnvrMcDtMlfT7BOdSAldvUwFz139/OTSkMjmJj/SSki3rZl kxp+QKT8+ZBLUaPnFv5cNkmrvK40+v5GaF5qqMbx3k0P5wbnsoO1q19OVx6mbnHi8LH1 H41PlmK6UrUV9FyIdUSVwSpuwxsHCuK53ZydWiermRdvXO+zx5wspUVa7CAC3b08t0JU mjFg== MIME-Version: 1.0 X-Received: by 10.194.77.179 with SMTP id t19mr73408789wjw.30.1436819673425; Mon, 13 Jul 2015 13:34:33 -0700 (PDT) Received: by 10.28.21.134 with HTTP; Mon, 13 Jul 2015 13:34:33 -0700 (PDT) In-Reply-To: References: Date: Mon, 13 Jul 2015 23:34:33 +0300 Message-ID: Subject: Re: [GSOC] bhyve port on ARM - weekly status report From: Mihai Carabas To: soc-status@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jul 2015 20:34:35 -0000 Hi everyone, >> In the last week I've started writing low-level ASM code which is > saving/restoring the guest/host state. I've created a logic for a full > pass from the host to the guest and back [1]: > - when the host is calling the "HYP" instruction, causes a trap in > hyp-mode where it can run different functions specific to this mode. An > example is hyp_enter_guest function which saves the host state, loads the > guest state and jumps to the guest pc with an eret instruction (back to svc > mode basically) > - when the guest executes an instruction that causes a trap (e.g. wfi > which was configured by us) the code is saving the guest state, loading the > host state and than is returning to host svc-mode with the exception of the > guest to manage it > - I've also implemented a call to panic function in host if the abort > exceptions in hyp-mode are raised. > > There are still some configuration registers that need to be tackled (like > VCTBR - defines the guest configuration for the stage 2 address > translation). > > In the next week I have to write the C code which is calling the ASM > methods in arm_vmrun and the init code which allocates the context > structure (arm_vminit) and maps it in HYP-Mode. I will try to integrate > these operations with bhyve userspace tools to ease the development. > > In the last week I've written the glue C code which is calling the ASM methods to run a VM. The logic flow starts from userspace from bhyvearm program (it's a duplicate of the normal bhyve due to its high dependency of x86) using ioctl's to run the VM. I've also written a simplified bhyveloadarm which maps the desired memory from the GPA indicated as a parameter (here I had to write the entire flow from userspace to kernel because there were empty functions - the MMAP ioctl). Than it loads the raw kernel image in the memory at a given address. In the end I set up the PC using also an ioctl. I've created a new ramdisk containing the bhyvearm* programs and the dynamic libraries they depend on. Then I've managed to make bhyveloadarm to copy a file which contains the bytecode for the "wfi" instruction in the guest memory and now I'm playing with bhyvearm to make the "guest" run that instruction. I'm investigating a bug in restoring the host state after the VM has run (the VM throws a data abort exception because I haven't configured the VTCR yet and I'm trying to send this exception back in bhyvearm userspace). Thanks, Mihai