From owner-svn-src-all@freebsd.org Thu Nov 30 19:38:55 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 880A8E69DD1; Thu, 30 Nov 2017 19:38:55 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-it0-f47.google.com (mail-it0-f47.google.com [209.85.214.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5829171B10; Thu, 30 Nov 2017 19:38:55 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-it0-f47.google.com with SMTP id m11so2135983iti.1; Thu, 30 Nov 2017 11:38:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=EChsBkOPtY1DLJH7Ew+dJv/5ZH07d1UZKOKCcxNGMO8=; b=OI9l9rjor7s+XxDKL96z5svIfZ6IC7gSSVk26OsAfPuAVx4LXT4Rudncv7rAxBUYxT fCKvt1Grs2L/jPaKDdUUKbTFX9dmmHvhqWjEpPVj07qKYxji5atdafB2Gn8fyeIdBiax 8GWmWF34MKWC4AR1No1pLXUTciezCda+RWbBmLYj7tfCOzL7mnhMHcDrQnvkjMnskEeb RJLPTIbZYvsEYt1fvzr79s3oYAup0qNk+J1lbNwgf9jo8CKaSyjMZ5eTAwDigmCmg5f3 U2q58Ks/0Ckk6paouV0pJqY8H15eT9Ay6uCZuRw6sA6XAHzuOAk4tzqug6etK59shEmS SyiQ== X-Gm-Message-State: AJaThX5KxeanTjqeW+LCLKiS4AZ+/cs6J284IHMUCEsbzNKbiuOrPoji /SYxNZy8V2ebCeXicbZyTc11GXi5 X-Google-Smtp-Source: AGs4zMbXo3M4rXiF4Y8DKBDWA5pVqW0vWZeZQwYBlN29HvuUwW0jGuKOnjIHppQLrEZhhAhp1UMZHg== X-Received: by 10.36.135.5 with SMTP id f5mr4778086ite.85.1512070350049; Thu, 30 Nov 2017 11:32:30 -0800 (PST) Received: from mail-io0-f179.google.com (mail-io0-f179.google.com. [209.85.223.179]) by smtp.gmail.com with ESMTPSA id s67sm2209427iod.85.2017.11.30.11.32.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Nov 2017 11:32:29 -0800 (PST) Received: by mail-io0-f179.google.com with SMTP id d16so8744713iob.4; Thu, 30 Nov 2017 11:32:29 -0800 (PST) X-Received: by 10.107.181.147 with SMTP id e141mr3255414iof.117.1512070349509; Thu, 30 Nov 2017 11:32:29 -0800 (PST) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.2.165.150 with HTTP; Thu, 30 Nov 2017 11:32:29 -0800 (PST) In-Reply-To: References: <201711300140.vAU1e7dC001292@repo.freebsd.org> From: Conrad Meyer Date: Thu, 30 Nov 2017 11:32:29 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r326383 - head/sys/x86/cpufreq To: Jung-uk Kim Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2017 19:38:55 -0000 Hi, I don't think this answers the second question about the conditional. It seems like PCPU_GET() for the initial CPU should be pulled out of the loop, which binds the thread to a different CPU every iteration. Also, as a side effect of disabling verification, you have fixed PR 221621, 219213, and probably 218262. Best, Conrad On Thu, Nov 30, 2017 at 11:21 AM, Jung-uk Kim wrote: > On 11/30/2017 13:57, Conrad Meyer wrote: >> Hi Jung-uk, >> >> I have some questions about this change. >> >> On Wed, Nov 29, 2017 at 5:40 PM, Jung-uk Kim wrote: >>> Author: jkim >>> Date: Thu Nov 30 01:40:07 2017 >>> New Revision: 326383 >>> URL: https://svnweb.freebsd.org/changeset/base/326383 >>> >>> Log: >>> Add a tunable "debug.hwpstate_verify" to check P-state after changing it and >>> turn it off by default. It is very inefficient to verify current P-state of >>> each core, especially for CPUs with many cores. When multiple commands are >>> requested to the same power domain before completion of pending transitions, >>> the last command is executed according to the manual. Because requests are >>> serialized by the caller, all cores will receive the same command for each >>> call. Do not call sched_bind() and sched_unbind(). It is redundant because >>> the caller does it anyway. >>> ... >>> @@ -176,47 +178,57 @@ hwpstate_goto_pstate(device_t dev, int pstate) >>> if (limit > id) >>> id = limit; >>> >> >> Should we bind the thread and record PCPU_GET() here? >> >>> + HWPSTATE_DEBUG(dev, "setting P%d-state on cpu%d\n", id, >>> + PCPU_GET(cpuid)); >>> + /* Go To Px-state */ >>> + wrmsr(MSR_AMD_10H_11H_CONTROL, id); >>> + >>> /* >>> * We are going to the same Px-state on all cpus. >>> * Probably should take _PSD into account. >>> */ >>> - error = 0; >>> CPU_FOREACH(i) { >>> + if (i == PCPU_GET(cpuid)) >> >> It seems like this check could evaluate to a different CPU every time? >> When really we are trying to avoid setting on the CPU we set on >> initially above? >> >>> + continue; >>> + >>> /* Bind to each cpu. */ >>> thread_lock(curthread); >>> sched_bind(curthread, i); >>> thread_unlock(curthread); >>> - HWPSTATE_DEBUG(dev, "setting P%d-state on cpu%d\n", >>> - id, PCPU_GET(cpuid)); >>> + HWPSTATE_DEBUG(dev, "setting P%d-state on cpu%d\n", id, i); >>> /* Go To Px-state */ >>> wrmsr(MSR_AMD_10H_11H_CONTROL, id); >>> } >>> - CPU_FOREACH(i) { >>> - /* Bind to each cpu. */ >>> - thread_lock(curthread); >>> - sched_bind(curthread, i); >>> - thread_unlock(curthread); >>> - /* wait loop (100*100 usec is enough ?) */ >>> - for (j = 0; j < 100; j++) { >>> - /* get the result. not assure msr=id */ >>> - msr = rdmsr(MSR_AMD_10H_11H_STATUS); >>> - if (msr == id) >>> - break; >>> - sbt = SBT_1MS / 10; >>> - tsleep_sbt(dev, PZERO, "pstate_goto", sbt, >>> - sbt >> tc_precexp, 0); >>> + >>> + /* >>> + * Verify whether each core is in the requested P-state. >>> + */ >>> + if (hwpstate_verify) { >>> + CPU_FOREACH(i) { >>> + thread_lock(curthread); >>> + sched_bind(curthread, i); >>> + thread_unlock(curthread); >>> + /* wait loop (100*100 usec is enough ?) */ >>> + for (j = 0; j < 100; j++) { >>> + /* get the result. not assure msr=id */ >>> + msr = rdmsr(MSR_AMD_10H_11H_STATUS); >>> + if (msr == id) >>> + break; >>> + sbt = SBT_1MS / 10; >>> + tsleep_sbt(dev, PZERO, "pstate_goto", sbt, >>> + sbt >> tc_precexp, 0); >>> + } >>> + HWPSTATE_DEBUG(dev, "result: P%d-state on cpu%d\n", >>> + (int)msr, i); >>> + if (msr != id) { >>> + HWPSTATE_DEBUG(dev, >>> + "error: loop is not enough.\n"); >> >> In this error return, should the current thread be unbinded? The old >> code did this by setting error and falling through to the ordinary >> exit path. We could use 'goto out;' to avoid looping through the rest >> of the CPUs. >> >>> + return (ENXIO); >>> + } >>> } >>> - HWPSTATE_DEBUG(dev, "result: P%d-state on cpu%d\n", >>> - (int)msr, PCPU_GET(cpuid)); >>> - if (msr != id) { >>> - HWPSTATE_DEBUG(dev, "error: loop is not enough.\n"); >>> - error = ENXIO; >>> - } >>> } >>> - thread_lock(curthread); >>> - sched_unbind(curthread); >>> - thread_unlock(curthread); >>> - return (error); >>> + >>> + return (0); >>> } >>> >>> static int >>> > > This driver is only called via cpufreq(4) (i.e., sys/kern/kern_cpu.c), > and sched_bind()/sched_unbind() is done from cf_set_method() for cpu0. > If you want to see the sequence, try "sysctl debug.cpufreq.verbose=1" > and "sysctl debug.hwpstate_verbose=1". > > Jung-uk Kim >