From owner-soc-status@FreeBSD.ORG Mon Jul 21 11:22:24 2014 Return-Path: Delivered-To: soc-status@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11D765C9 for ; Mon, 21 Jul 2014 11:22:24 +0000 (UTC) Received: from mail-qc0-x232.google.com (mail-qc0-x232.google.com [IPv6:2607:f8b0:400d:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C65BB27A9 for ; Mon, 21 Jul 2014 11:22:23 +0000 (UTC) Received: by mail-qc0-f178.google.com with SMTP id x3so5393772qcv.23 for ; Mon, 21 Jul 2014 04:22:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=RjQ0jFQ0l7h+cXQ1IsY7p1NiYY4j8/tA++Ot8RRamXA=; b=QHpv/UZeYlGTxmQPkNiIKD4DYdk98zEI7SDXBM4+VPEo3mdBchE41JjpvoPxGzpUeu uQc3jmsLagMLqzp0pd13bdw94Ni3S23Z8gaemVGcTSMVW8sftv7XQScEVHON4fULwfSu q5i+1cbOan+8UxUnIbYUnIQZ5xpoBO0vsjNFFNfPJ6Nh1SBof5b/2tYxyBIc1hmflNb1 FORbbjBkbCcH3lsvhwlpI99vtHRJ+Tixx91TFiNGCVASu4dd09Ry4UKSFt5NrfeZYhrg OJjtMOkk6Q46LfiQw4hMKlKruwDaCbVn8VVctqXOBu9AC9qQ7q3dc5KUFKyRSqxdST+j OSTw== MIME-Version: 1.0 X-Received: by 10.224.131.8 with SMTP id v8mr4477799qas.31.1405941742929; Mon, 21 Jul 2014 04:22:22 -0700 (PDT) Received: by 10.140.84.85 with HTTP; Mon, 21 Jul 2014 04:22:22 -0700 (PDT) In-Reply-To: References: Date: Mon, 21 Jul 2014 14:22:22 +0300 Message-ID: Subject: Re: [GSOC] bhyve instruction caching From: Mihai Carabas To: soc-status@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 11:22:24 -0000 > >> >> Also I'm looking for new real-world benchmarks to see where we can >> benefict a lot from the instruction caching feature. > I've started doing to kind of benchmarks: > * CPU intensive (ex: a bash script that increments in a loop a variable) > * benchmarks that creates lot of short processes (not CPU intensive > but short running I/O intensive): make buildworld -j2 > > I will come with some concrete results in the next week. Meanwhile I > will add some new statistics like "Cache hits" and "Cache evictions" > to see how my cache behavious in each of the benchmarks. I have added the statistics about cache hits and evictions using the vmm_stat_* infrastructure. I added three type of variables: - VM_INST_CACHE_HITS - the number of cache hits - VM_INST_CACHE_INSERTIONS - the number of cache insertions - VM_INST_CACHE_EVICTIONS[4] - an array with cache evictions corresponding to each level of a page table (basically the pagetable which caused the eviction) 1) CPU intensive (simple bash script) with 2 VCPUs """ #!/usr/local/bin/bash a=0 while true do a=$((a+1)) done """ After 11 minutes of running here are the results: VCPU0: number of instruction cache hits 699519 number of instruction cache evictions[0] 7139 number of instruction cache evictions[1] 0 number of instruction cache evictions[2] 0 number of instruction cache evictions[3] 0 number of instruction cache insertions 10395 VCPU1: number of instruction cache hits 840485 number of instruction cache evictions[0] 8926 number of instruction cache evictions[1] 0 number of instruction cache evictions[2] 0 number of instruction cache evictions[3] 0 number of instruction cache insertions 5743 As you can see the number of evictions is very low for this kind of process 2) "make buildworld -j2" with 2 VCPUs: vcpu:0 number of instruction cache hits 19204630 number of instruction cache evictions[0] 8563694 number of instruction cache evictions[1] 1131 number of instruction cache evictions[2] 0 number of instruction cache evictions[3] 0 number of instruction cache insertions 8688733 vcpu:1 number of instruction cache hits 12930500 number of instruction cache evictions[0] 9173381 number of instruction cache evictions[1] 1457 number of instruction cache evictions[2] 0 number of instruction cache evictions[3] 0 number of instruction cache insertions 9051295 As you can see in this case, the cache hit is much lower due to the high-rate eviction caused mainly by the PML4 entry. This is caused by the make infrastructure which is creating/deleting a lot of small processes. This is the analysis about the cache hit metric. The code writing for this project is at the end and I will come at the end of the week with some high-level measurements, comparing the running times with the non-cache implementation. Thanks, Mihai