From owner-freebsd-stable@FreeBSD.ORG Wed Nov 19 22:15:08 2014
Return-Path: <owner-freebsd-stable@FreeBSD.ORG>
Delivered-To: freebsd-stable@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
[IPv6:2001:1900:2254:206a::19:1])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by hub.freebsd.org (Postfix) with ESMTPS id 4CD5E582;
Wed, 19 Nov 2014 22:15:08 +0000 (UTC)
Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com
[IPv6:2a00:1450:4010:c03::22f])
(using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G2" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 9ED8DDF6;
Wed, 19 Nov 2014 22:15:07 +0000 (UTC)
Received: by mail-la0-f47.google.com with SMTP id hz20so1366725lab.34
for <multiple recipients>; Wed, 19 Nov 2014 14:15:05 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:cc:content-type;
bh=Ivbgx+ZC0LED4idJVKXuOvJtDLukpnN/yy/aKyWyHus=;
b=BH2fWoHolYI+rYJRDLijQE5F5WdTzcjjino8Fc2YGjTOOqI7PIbM5YG6FRv9W9Uvcw
/vI2cYVTlpeQmFmILYxyBTujojlwDB84v7DmqGiX/5O7Lh+5k6fqYSHxZXrNUDmR/JrZ
X8qyHZpMtY2ancIzTid+bpK1igRiXaNo1ZoSQoGzDNc2XTxRCRxkm6i98cQcIlG9M8/8
dw+JgiZ9uoAOi35RtBpjOs4HRPzAV5pwNnQ4tkAARZ2EeynSvy0tuWaIbChx8HNqMLkA
LeCrqY7i+CLRek4i550j8ylsGt7mfAFy5Ez/0/vp4jyyDB0DgAR9o/weRAORhcP9yj1y
fFNg==
MIME-Version: 1.0
X-Received: by 10.112.130.132 with SMTP id oe4mr4474294lbb.82.1416435305611;
Wed, 19 Nov 2014 14:15:05 -0800 (PST)
Received: by 10.25.170.66 with HTTP; Wed, 19 Nov 2014 14:15:05 -0800 (PST)
In-Reply-To: <3870083.h75yLjv6As@ralph.baldwin.cx>
References: <alpine.BSF.2.00.1411112022500.25949@woozle.rinet.ru>
<CAPS9+SujGtrQ+w=5jCBqV1vv_ZFZ5qkhjfGQUwc5n0sfjPCvnQ@mail.gmail.com>
<CAPS9+SuCLE9kVPA-RNMHfziJYXAL1ivkV-B8wa64athYdDb38A@mail.gmail.com>
<3870083.h75yLjv6As@ralph.baldwin.cx>
Date: Wed, 19 Nov 2014 23:15:05 +0100
Message-ID: <CAPS9+SuP+i6W5VHkx7gaec=QqKimEwGd1t30sdz8zN6WVcCGiQ@mail.gmail.com>
Subject: Re: SuperMicro IPMI/SOL and ipmitool troubles
From: Andreas Nilsson <andrnils@gmail.com>
To: John Baldwin <jhb@freebsd.org>
Content-Type: text/plain; charset=UTF-8
X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1
Cc: Daniel O'Connor <doconnor@gsoft.com.au>,
FreeBSD Stable Mailing List <freebsd-stable@freebsd.org>,
Dmitry Morozovsky <marck@rinet.ru>
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-stable>,
<mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
<mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Nov 2014 22:15:08 -0000
On Wed, Nov 19, 2014 at 8:05 PM, John Baldwin <jhb@freebsd.org> wrote:
> On Wednesday, November 19, 2014 05:02:49 PM Andreas Nilsson wrote:
> > On Wed, Nov 19, 2014 at 3:28 PM, Andreas Nilsson <andrnils@gmail.com>
> wrote:
> > > On Fri, Nov 14, 2014 at 7:30 PM, Dmitry Morozovsky <marck@rinet.ru>
> wrote:
> > >> Daniel,
> > >>
> > >> nice to see you here too ;)
> > >>
> > >> On Fri, 14 Nov 2014, Daniel O'Connor wrote:
> > >> > On 12 Nov 2014, at 19:43, Andreas Nilsson <andrnils@gmail.com>
> wrote:
> > >> > > unclear is the word for it :) And thanks for looking into this.
> > >>
> > >> ipmi/ilo is
> > >>
> > >> > > important on a server os.
> > >>
> > >> > > I found a reference to it in a ML post:
> > >>
> http://lists.freebsd.org/pipermail/freebsd-stable/2013-February/072464.ht
> > >> ml
> > >>
> > >> > I started that thread :)
> > >> > I did get it working on the hardware I was using (Supermicro X9SCL-F
> > >>
> > >> and X8SIL-F)
> > >>
> > >> > I used the following BIOS settings
> > >> >
> > >> > ? Remote Access - Enabled
> > >> > ? Serial Port Number - COM3
> > >> > ? Serial Port Mode - 115200, 8, n, 1
> > >> > ? Flow Control - Hardware
> > >> > ? Redirection After BIOS POST - Always
> > >> > ? Terminal Type - VT100
> > >> > ? VT-UTF8 Combo Key Support - Disabled
> > >> > ? Sredir Memory Display Delay - No Delay
> > >> >
> > >> > And the following in loader.conf
> > >> > # Give preference to VGA console
> > >> > console="vidconsole,comconsole"
> > >> > # Uncomment below and comment above to give serial console
> preference
> > >> > #console="comconsole,vidconsole"
> > >> > comconsole_speed="115200"
> > >> > boot_multicons="YES"
> > >> > hint.uart.0.flags="0x0"
> > >> > hint.uart.2.at="isa"
> > >> > hint.uart.2.port="0x3E8"
> > >> > hint.uart.2.flags="0x30"
> > >> >
> > >> > And this in /etc/ttys
> > >> > # IPMI console
> > >> > # Note: The Java console viewer doesn't seem to be very smart as it
> > >>
> > >> doesn't
> > >>
> > >> > # properly support VT100
> > >> > cuau2 "/usr/libexec/getty 3wire.115200" vt100 on secure
> > >> >
> > >> > I could then access it using ipmitool like so
> > >> > ipmitool -H remoteip -U ADMIN -I lanplus sol activate
> > >> > [login]
> > >> > export TERM=xterm
> > >> >
> > >> > Note that I wanted vidconsole by default because mostly the systems
> > >>
> > >> were used by people local to them, however we could break into the
> loader
> > >> and type 'set console=comconsole,vidconsole? and then get everything
> over
> > >> the serial console for remote trouble shooting.
> > >>
> > >> > You may also wish to check the IPMI configuration via the web
> interface
> > >>
> > >> - by default it will failover to port 0 and it has terrible default
> > >> passwords. I changed the passwords and forced it to use the dedicated
> > >> IPMI
> > >> port even if nothing was connected to it.
> > >>
> > >> Well, I'm almost done with most of our SM server, even concentrated
> > >> console on
> > >> our console server with such a simple config:
> > >>
> > >> ---- 8< ----
> > >> # ipmi/sol console template
> > >> default ipmi {
> > >>
> > >> master localhost;
> > >> type exec;
> > >> exec /usr/local/bin/ipmitool -f /usr/local/etc/ipmi-pass -U
> > >>
> > >> root -I lanplus -H %.int sol activate;
> > >>
> > >> execsubst %=cs;
> > >> #idletimeout 6h;
> > >>
> > >> break 0 { string "~B"; }
> > >>
> > >> }
> > >>
> > >> console gwn1 { include ipmi; }
> > >> console gwn2 { include ipmi; }
> > >> console gwn3 { include ipmi; }
> > >> console gwn4 { include ipmi; }
> > >> console gwn5 { include ipmi; }
> > >> console gwn6 { include ipmi; }
> > >> console gwn7 { include ipmi; }
> > >> console gwn8 { include ipmi; }
> > >>
> > >> console gwc2 { include ipmi; }
> > >> ---- 8< ----
> > >>
> > >> This has console logging (including possible panics) as a surplus
> > >>
> > >> --
> > >> Sincerely,
> > >> D.Marck [DM5020, MCK-RIPE,
> DM3-RIPN]
> > >> [ FreeBSD committer: marck@FreeBSD.org
> ]
> > >>
> ------------------------------------------------------------------------
> > >> *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru
> ***
> > >>
> ------------------------------------------------------------------------
> > >
> > > Hello again,
> > >
> > > Searching on hw.uart.console, I found:
> > >
> http://lists.freebsd.org/pipermail/svn-src-head/2013-February/044641.html
> > > , a very enlightening thread.
> > >
> > > Basically: "ohh, you want to use something other than COM1 and tried to
> > > get away with just changing hint.uart stuff, which has worked for a
> while,
> > > ha, no way..." No heads up, nothing.
> > >
> > > Sorry to say jhb@ but is not a rare case. It is if not the default, a
> > > very common setup on every HP server with iLO, and it holds for most
> all
> > > OOB style serial emulation I have ever had the (dis)pleasure of working
> > > with.
>
> This was done _specifically_ so you could use non-COM1 for both loader and
> kernel with one thing to change. That is, you don't use hint.uart.X.flags
> after this. I have used this with many SuperMicro servers that use COM2
> and
> COM3 because I wanted the entire path (boot loader and kernel) to work, not
> the kernel only. Having only the kernel means I can't break into the
> loader
> prompt to boot a different kernel, single user, etc.
>
> To clarify, are you using _different_ serial ports for the loader vs the
> kernel? That is the use case I considered to be rare. Every single server
> I have ever worked with (though not iLO, mostly Dell and SuperMicro) uses
> the
> same COM port for serial redirection rather for SOL or via actual cables.
> I've yet to use a system that, for example, used COM1 for the loader and
> COM2
> for the kernel. You are saying that every HP server uses COM1 for the
> loader
> and COM2 for the kernel?
>
> --
> John Baldwin
>
Hello,
For Supermicro:
The box was in a rack when I started this, so I don't really know how the
bios is setup, but as far as I can gather, getting
"BTX loader yadda yadda" seems to always happen, on any COM-port.
The actual output from loading modules and so on only happens with specific
settings. Here ( as in an answer to Dmitry I accidentally sent in private )
things seems to be on different COMs: Setting: comconsole_port=0x2F8 (com2)
show loader and goes dark on handoff to kernel
comconsole_port=0x3E8 (com3) show no loader and stays dark
Neither of them gives me console even after successful boot. The only way
to get serial is setting hw.uart.console="br:9600" explicitly.
For HP
I asked a colleague, and we couldn't exactly remember ( it was quite a
while since these boxes were deployed ), but our recollection was that
serial setup was left as-is, which seems to be that "vsp" from iLO gives
you a tty on 0x2F8. Setting that in comconsole_port in loader.conf works.
Both
In an effort of standardising stuff, we put hw-dependent stuff in
loader.conf, ie hint.uart redirection to have the same entry in the ttys
file.
Having serial over ipmi/ilo go dark during remote upgrades is scary at
best. Seeing that this change has such a potential, how come it is not
mentioned in the release notes?
Best regards
Andreas