Date: Sun, 6 Jul 2014 14:59:41 +0300 From: Mihai Carabas <mihai.carabas@gmail.com> To: soc-status@freebsd.org Subject: Re: [GSOC] bhyve instruction caching Message-ID: <CANg1yUtcBz3OiL=R-n=Kh1o68-tECLR3FA0ag23F5Z9CjJrFjA@mail.gmail.com> In-Reply-To: <CANg1yUsmRCSftYgFWZu_xu-nCROMn3FvuXzfgteiuy4LtAJtvQ@mail.gmail.com> References: <CANg1yUuazrhybHVVzi2g8vCBSTx3Z=gYmEVXvEMuj2SN%2BRY9Sg@mail.gmail.com> <CANg1yUu_b0qSX=2eXRaO31cogjGdSmkDnEh7PAjfVvCMsAaC1g@mail.gmail.com> <CANg1yUuZU0--O8RgOVx=jKhku1yguvmO4TxUZ5c4wEq6jk6fSw@mail.gmail.com> <CANg1yUsmRCSftYgFWZu_xu-nCROMn3FvuXzfgteiuy4LtAJtvQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Jun 15, 2014 at 8:58 PM, Mihai Carabas <mihai.carabas@gmail.com> wrote: > On Sun, Jun 8, 2014 at 12:38 AM, Mihai Carabas <mihai.carabas@gmail.com> wrote: >>> >>> These days I've started a discussion with Neel about some >>> microbenchmarking mechanisms. I will come with some more details next >>> week. >> I've built a microbenchmarking kernel module which is accessing the >> lapic->id for 1000000 and than I calculate the average of an access >> (each access needs to be emulated by the hypervisor). >> >> I've also implemented the instuction caching mechanism. At each emulation: >> - I check to see if I have that particular instruction cached >> - if not I will cache it in a particula structure named "struct vie_cached" [1] >> - if it's cached I just use that instruction >> >> Right now I am working on write-protecting the pages where the >> instruction reside. I will come with some more details/results when I >> finished this part too (there are some SMP issues I'm still debating >> with Neel). > > I added the write protection for the pages where instruction resides > (I mark them with PROT_READ|PROT_EXECUTE using vm_map_protect). When a > page fault is raised on that page I delete the instruction from the > cache. I've tested the instruction caching feature and it works ok (no > bugs reveal). The results on the microbenchmarking are looking OK, but > they aren't conclusive because I'm not write protecting the pages of > the pagetable (which will eat more time). Right now for 1000000 > accesses to the lapic-id we have an average of 6200 ticks per VM_EXIT > for an instruction emulation. Without instruction caching the average > is at about 10000ticks. Like I've said before, this results aren't > very conclusive until I introduce all protections needed. I will come > in the next weeks with more results when I have the caching feature > fully implemented. > > I've also introduce a new systl to enable/disable instruction caching > "hw.vmm.instruction_cache". In the last two weeks I developed the mechanism of write-protecting the VM pagetable: every page of the pagetable that was pointing to the memory page where the instruction resides was made read-only. I had some trouble with the locking mechanism when having a virtual machine with more than 1 vCPU (one vCPU was write protecting and immediatly another VCPU was remove the protection resulting in a corruption of the write-protection and cache logic). Neel helped me solve this issue with some best effort checks and are working OK. Also I migrated to sx_* locks due to the fact the rm_* locks didn' let me sleep in them (at some point when raising the protection in the VM subsystem a sleep may be encountered). At the end of the week I will come with some new results with the microbenchmarking. Also I'm looking for new real-world benchmarks to see where we can benefict a lot from the instruction caching feature. Thanks, Mihai
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANg1yUtcBz3OiL=R-n=Kh1o68-tECLR3FA0ag23F5Z9CjJrFjA>