From owner-svn-src-all@FreeBSD.ORG Tue Mar 25 19:20:35 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5E1558E; Tue, 25 Mar 2014 19:20:35 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B17489D5; Tue, 25 Mar 2014 19:20:35 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s2PJKZis047902; Tue, 25 Mar 2014 19:20:35 GMT (envelope-from tychon@svn.freebsd.org) Received: (from tychon@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s2PJKYnX047892; Tue, 25 Mar 2014 19:20:34 GMT (envelope-from tychon@svn.freebsd.org) Message-Id: <201403251920.s2PJKYnX047892@svn.freebsd.org> From: Tycho Nightingale Date: Tue, 25 Mar 2014 19:20:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r263744 - in head: sys/amd64/include sys/amd64/vmm sys/amd64/vmm/io sys/modules/vmm usr.sbin/bhyve X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 19:20:35 -0000 Author: tychon Date: Tue Mar 25 19:20:34 2014 New Revision: 263744 URL: http://svnweb.freebsd.org/changeset/base/263744 Log: Move the atpit device model from userspace into vmm.ko for better precision and lower latency. Approved by: grehan (co-mentor) Added: head/sys/amd64/vmm/io/vatpit.c (contents, props changed) head/sys/amd64/vmm/io/vatpit.h (contents, props changed) Deleted: head/usr.sbin/bhyve/pit_8254.c head/usr.sbin/bhyve/pit_8254.h Modified: head/sys/amd64/include/vmm.h head/sys/amd64/vmm/vmm.c head/sys/amd64/vmm/vmm_ioport.c head/sys/modules/vmm/Makefile head/usr.sbin/bhyve/Makefile Modified: head/sys/amd64/include/vmm.h ============================================================================== --- head/sys/amd64/include/vmm.h Tue Mar 25 19:17:22 2014 (r263743) +++ head/sys/amd64/include/vmm.h Tue Mar 25 19:20:34 2014 (r263744) @@ -191,6 +191,7 @@ struct vmspace *vm_get_vmspace(struct vm int vm_assign_pptdev(struct vm *vm, int bus, int slot, int func); int vm_unassign_pptdev(struct vm *vm, int bus, int slot, int func); struct vatpic *vm_atpic(struct vm *vm); +struct vatpit *vm_atpit(struct vm *vm); /* * Inject exception 'vme' into the guest vcpu. This function returns 0 on Added: head/sys/amd64/vmm/io/vatpit.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/amd64/vmm/io/vatpit.c Tue Mar 25 19:20:34 2014 (r263744) @@ -0,0 +1,370 @@ +/*- + * Copyright (c) 2014 Tycho Nightingale + * Copyright (c) 2011 NetApp, Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY NETAPP, INC ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL NETAPP, INC OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "vmm_ktr.h" +#include "vatpic.h" +#include "vioapic.h" +#include "vatpit.h" + +static MALLOC_DEFINE(M_VATPIT, "atpit", "bhyve virtual atpit (8254)"); + +#define VATPIT_LOCK(vatpit) mtx_lock_spin(&((vatpit)->mtx)) +#define VATPIT_UNLOCK(vatpit) mtx_unlock_spin(&((vatpit)->mtx)) +#define VATPIT_LOCKED(vatpit) mtx_owned(&((vatpit)->mtx)) + +#define TIMER_SEL_MASK 0xc0 +#define TIMER_RW_MASK 0x30 +#define TIMER_MODE_MASK 0x0f +#define TIMER_SEL_READBACK 0xc0 + +#define PIT_8254_FREQ 1193182 +#define TIMER_DIV(freq, hz) (((freq) + (hz) / 2) / (hz)) + +struct vatpit_callout_arg { + struct vatpit *vatpit; + int channel_num; +}; + + +struct channel { + int mode; + uint16_t initial; /* initial counter value */ + sbintime_t now_sbt; /* uptime when counter was loaded */ + uint8_t cr[2]; + uint8_t ol[2]; + int crbyte; + int olbyte; + int frbyte; + struct callout callout; + sbintime_t callout_sbt; /* target time */ + struct vatpit_callout_arg callout_arg; +}; + +struct vatpit { + struct vm *vm; + struct mtx mtx; + + sbintime_t freq_sbt; + + struct channel channel[3]; +}; + +#define VATPIT_CTR0(vatpit, fmt) \ + VM_CTR0((vatpit)->vm, fmt) + +#define VATPIT_CTR1(vatpit, fmt, a1) \ + VM_CTR1((vatpit)->vm, fmt, a1) + +#define VATPIT_CTR2(vatpit, fmt, a1, a2) \ + VM_CTR2((vatpit)->vm, fmt, a1, a2) + +#define VATPIT_CTR3(vatpit, fmt, a1, a2, a3) \ + VM_CTR3((vatpit)->vm, fmt, a1, a2, a3) + +#define VATPIT_CTR4(vatpit, fmt, a1, a2, a3, a4) \ + VM_CTR4((vatpit)->vm, fmt, a1, a2, a3, a4) + +static void pit_timer_start_cntr0(struct vatpit *vatpit); + +static void +vatpit_callout_handler(void *a) +{ + struct vatpit_callout_arg *arg = a; + struct vatpit *vatpit; + struct callout *callout; + struct channel *c; + + vatpit = arg->vatpit; + c = &vatpit->channel[arg->channel_num]; + callout = &c->callout; + + VATPIT_CTR1(vatpit, "atpit t%d fired", arg->channel_num); + + VATPIT_LOCK(vatpit); + + if (callout_pending(callout)) /* callout was reset */ + goto done; + + if (!callout_active(callout)) /* callout was stopped */ + goto done; + + callout_deactivate(callout); + + if (c->mode == TIMER_RATEGEN) { + pit_timer_start_cntr0(vatpit); + } + + vatpic_pulse_irq(vatpit->vm, 0); + vioapic_pulse_irq(vatpit->vm, 2); + +done: + VATPIT_UNLOCK(vatpit); + return; +} + +static void +pit_timer_start_cntr0(struct vatpit *vatpit) +{ + struct channel *c; + sbintime_t delta, precision; + + c = &vatpit->channel[0]; + if (c->initial != 0) { + delta = c->initial * vatpit->freq_sbt; + precision = delta >> tc_precexp; + c->callout_sbt = c->callout_sbt + delta; + + callout_reset_sbt(&c->callout, c->callout_sbt, + precision, vatpit_callout_handler, &c->callout_arg, + C_ABSOLUTE); + } +} + +static uint16_t +pit_update_counter(struct vatpit *vatpit, struct channel *c, bool latch) +{ + uint16_t lval; + sbintime_t delta_ticks; + + /* cannot latch a new value until the old one has been consumed */ + if (latch && c->olbyte != 0) + return (0); + + if (c->initial == 0) { + /* + * This is possibly an o/s bug - reading the value of + * the timer without having set up the initial value. + * + * The original user-space version of this code set + * the timer to 100hz in this condition; do the same + * here. + */ + c->initial = TIMER_DIV(PIT_8254_FREQ, 100); + c->now_sbt = sbinuptime(); + } + + delta_ticks = (sbinuptime() - c->now_sbt) / vatpit->freq_sbt; + + lval = c->initial - delta_ticks % c->initial; + + if (latch) { + c->olbyte = 2; + c->ol[1] = lval; /* LSB */ + c->ol[0] = lval >> 8; /* MSB */ + } + + return (lval); +} + +static int +vatpit_update_mode(struct vatpit *vatpit, uint8_t val) +{ + struct channel *c; + int sel, rw, mode; + + sel = val & TIMER_SEL_MASK; + rw = val & TIMER_RW_MASK; + mode = val & TIMER_MODE_MASK; + + if (sel == TIMER_SEL_READBACK) + return (-1); + + if (rw != TIMER_LATCH && rw != TIMER_16BIT) + return (-1); + + if (rw != TIMER_LATCH) { + /* + * Counter mode is not affected when issuing a + * latch command. + */ + if (mode != TIMER_INTTC && + mode != TIMER_RATEGEN && + mode != TIMER_SQWAVE && + mode != TIMER_SWSTROBE) + return (-1); + } + + c = &vatpit->channel[sel >> 6]; + if (rw == TIMER_LATCH) + pit_update_counter(vatpit, c, true); + else { + c->mode = mode; + c->olbyte = 0; /* reset latch after reprogramming */ + } + + return (0); +} + +static int +vatpit_get_out(struct vatpit *vatpit, int channel) +{ + struct channel *c; + sbintime_t delta_ticks; + int out; + + c = &vatpit->channel[channel]; + + switch (c->mode) { + case TIMER_INTTC: + delta_ticks = (sbinuptime() - c->now_sbt) / vatpit->freq_sbt; + out = ((c->initial - delta_ticks) <= 0); + break; + default: + out = 0; + break; + } + + return (out); +} + +int +vatpit_handler(void *vm, int vcpuid, struct vm_exit *vmexit) +{ + struct vatpit *vatpit; + struct channel *c; + int port; + uint8_t val; + int error; + + vatpit = vm_atpit(vm); + + if (vmexit->u.inout.bytes != 1) + return (-1); + + val = vmexit->u.inout.eax; + port = vmexit->u.inout.port; + + if (port == TIMER_MODE) { + if (vmexit->u.inout.in != 0) { + VATPIT_CTR0(vatpit, "vatpit attempt to read mode"); + return (-1); + } + + VATPIT_LOCK(vatpit); + error = vatpit_update_mode(vatpit, val); + VATPIT_UNLOCK(vatpit); + + return (error); + } + + /* counter ports */ + KASSERT(port >= TIMER_CNTR0 && vmexit->u.inout.port <= TIMER_CNTR2, + ("invalid port 0x%x", port)); + c = &vatpit->channel[port - TIMER_CNTR0]; + + VATPIT_LOCK(vatpit); + if (vmexit->u.inout.in) { + /* + * The spec says that once the output latch is completely + * read it should revert to "following" the counter. Use + * the free running counter for this case (i.e. Linux + * TSC calibration). Assuming the access mode is 16-bit, + * toggle the MSB/LSB bit on each read. + */ + if (c->olbyte == 0) { + uint16_t tmp; + + tmp = pit_update_counter(vatpit, c, false); + if (c->frbyte) + tmp >>= 8; + tmp &= 0xff; + vmexit->u.inout.eax = tmp; + c->frbyte ^= 1; + } else + vmexit->u.inout.eax = c->ol[--c->olbyte]; + } else { + c->cr[c->crbyte++] = vmexit->u.inout.eax; + if (c->crbyte == 2) { + c->frbyte = 0; + c->crbyte = 0; + c->initial = c->cr[0] | (uint16_t)c->cr[1] << 8; + c->now_sbt = sbinuptime(); + /* Start an interval timer for channel 0 */ + if (port == TIMER_CNTR0) { + c->callout_sbt = c->now_sbt; + pit_timer_start_cntr0(vatpit); + } + if (c->initial == 0) + c->initial = 0xffff; + } + } + VATPIT_UNLOCK(vatpit); + + return (0); +} + +struct vatpit * +vatpit_init(struct vm *vm) +{ + struct vatpit *vatpit; + struct bintime bt; + struct vatpit_callout_arg *arg; + int i; + + vatpit = malloc(sizeof(struct vatpit), M_VATPIT, M_WAITOK | M_ZERO); + vatpit->vm = vm; + + mtx_init(&vatpit->mtx, "vatpit lock", NULL, MTX_SPIN); + + FREQ2BT(PIT_8254_FREQ, &bt); + vatpit->freq_sbt = bttosbt(bt); + + for (i = 0; i < 3; i++) { + callout_init(&vatpit->channel[i].callout, true); + arg = &vatpit->channel[i].callout_arg; + arg->vatpit = vatpit; + arg->channel_num = i; + } + + return (vatpit); +} + +void +vatpit_cleanup(struct vatpit *vatpit) +{ + int i; + + for (i = 0; i < 3; i++) + callout_drain(&vatpit->channel[i].callout); + + free(vatpit, M_VATPIT); +} Added: head/sys/amd64/vmm/io/vatpit.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/amd64/vmm/io/vatpit.h Tue Mar 25 19:20:34 2014 (r263744) @@ -0,0 +1,40 @@ +/*- + * Copyright (c) 2014 Tycho Nightingale + * Copyright (c) 2011 NetApp, Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY NETAPP, INC ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL NETAPP, INC OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _VATPIT_H_ +#define _VATPIT_H_ + +#include + +struct vatpit *vatpit_init(struct vm *vm); +void vatpit_cleanup(struct vatpit *vatpit); + +int vatpit_handler(void *vm, int vcpuid, struct vm_exit *vmexit); + +#endif /* _VATPIT_H_ */ Modified: head/sys/amd64/vmm/vmm.c ============================================================================== --- head/sys/amd64/vmm/vmm.c Tue Mar 25 19:17:22 2014 (r263743) +++ head/sys/amd64/vmm/vmm.c Tue Mar 25 19:20:34 2014 (r263744) @@ -68,6 +68,7 @@ __FBSDID("$FreeBSD$"); #include "vmm_mem.h" #include "vmm_util.h" #include "vatpic.h" +#include "vatpit.h" #include "vhpet.h" #include "vioapic.h" #include "vlapic.h" @@ -119,6 +120,7 @@ struct vm { struct vhpet *vhpet; /* virtual HPET */ struct vioapic *vioapic; /* virtual ioapic */ struct vatpic *vatpic; /* virtual atpic */ + struct vatpit *vatpit; /* virtual atpit */ struct vmspace *vmspace; /* guest's address space */ struct vcpu vcpu[VM_MAXCPU]; int num_mem_segs; @@ -349,6 +351,7 @@ vm_create(const char *name, struct vm ** vm->vioapic = vioapic_init(vm); vm->vhpet = vhpet_init(vm); vm->vatpic = vatpic_init(vm); + vm->vatpit = vatpit_init(vm); for (i = 0; i < VM_MAXCPU; i++) { vcpu_init(vm, i); @@ -381,6 +384,7 @@ vm_destroy(struct vm *vm) if (vm->iommu != NULL) iommu_destroy_domain(vm->iommu); + vatpit_cleanup(vm->vatpit); vhpet_cleanup(vm->vhpet); vatpic_cleanup(vm->vatpic); vioapic_cleanup(vm->vioapic); @@ -1740,3 +1744,9 @@ vm_atpic(struct vm *vm) { return (vm->vatpic); } + +struct vatpit * +vm_atpit(struct vm *vm) +{ + return (vm->vatpit); +} Modified: head/sys/amd64/vmm/vmm_ioport.c ============================================================================== --- head/sys/amd64/vmm/vmm_ioport.c Tue Mar 25 19:17:22 2014 (r263743) +++ head/sys/amd64/vmm/vmm_ioport.c Tue Mar 25 19:20:34 2014 (r263744) @@ -36,11 +36,16 @@ __FBSDID("$FreeBSD$"); #include #include "vatpic.h" +#include "vatpit.h" #include "vmm_ioport.h" #define MAX_IOPORTS 1280 ioport_handler_func_t ioport_handler[MAX_IOPORTS] = { + [TIMER_MODE] = vatpit_handler, + [TIMER_CNTR0] = vatpit_handler, + [TIMER_CNTR1] = vatpit_handler, + [TIMER_CNTR2] = vatpit_handler, [IO_ICU1] = vatpic_master_handler, [IO_ICU1 + ICU_IMR_OFFSET] = vatpic_master_handler, [IO_ICU2] = vatpic_slave_handler, Modified: head/sys/modules/vmm/Makefile ============================================================================== --- head/sys/modules/vmm/Makefile Tue Mar 25 19:17:22 2014 (r263743) +++ head/sys/modules/vmm/Makefile Tue Mar 25 19:20:34 2014 (r263744) @@ -29,6 +29,7 @@ SRCS+= vmm.c \ SRCS+= iommu.c \ ppt.c \ vatpic.c \ + vatpit.c \ vhpet.c \ vioapic.c \ vlapic.c Modified: head/usr.sbin/bhyve/Makefile ============================================================================== --- head/usr.sbin/bhyve/Makefile Tue Mar 25 19:17:22 2014 (r263743) +++ head/usr.sbin/bhyve/Makefile Tue Mar 25 19:20:34 2014 (r263744) @@ -27,7 +27,6 @@ SRCS= \ pci_virtio_block.c \ pci_virtio_net.c \ pci_uart.c \ - pit_8254.c \ pm.c \ pmtmr.c \ post.c \