Date: Thu, 20 Nov 2014 13:51:29 -0500 From: John Baldwin <jhb@freebsd.org> To: Andreas Nilsson <andrnils@gmail.com> Cc: Daniel O'Connor <doconnor@gsoft.com.au>, FreeBSD Stable Mailing List <freebsd-stable@freebsd.org>, Dmitry Morozovsky <marck@rinet.ru> Subject: Re: SuperMicro IPMI/SOL and ipmitool troubles Message-ID: <201411201351.29190.jhb@freebsd.org> In-Reply-To: <CAPS9%2BSsU_Yimg6j980fB8by7HGRbBNxZ280yVArx4aV2nFynAQ@mail.gmail.com> References: <alpine.BSF.2.00.1411112022500.25949@woozle.rinet.ru> <201411201145.01162.jhb@freebsd.org> <CAPS9%2BSsU_Yimg6j980fB8by7HGRbBNxZ280yVArx4aV2nFynAQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thursday, November 20, 2014 1:09:06 pm Andreas Nilsson wrote: > On Thu, Nov 20, 2014 at 5:45 PM, John Baldwin <jhb@freebsd.org> wrote: > > > On Wednesday, November 19, 2014 5:15:05 pm Andreas Nilsson wrote: > > > On Wed, Nov 19, 2014 at 8:05 PM, John Baldwin <jhb@freebsd.org> wrote: > > > > > > > On Wednesday, November 19, 2014 05:02:49 PM Andreas Nilsson wrote: > > > > > On Wed, Nov 19, 2014 at 3:28 PM, Andreas Nilsson <andrnils@gmail.com > > > > > > > wrote: > > > > > > On Fri, Nov 14, 2014 at 7:30 PM, Dmitry Morozovsky <marck@rinet.ru > > > > > > > wrote: > > > > > >> Daniel, > > > > > >> > > > > > >> nice to see you here too ;) > > > > > >> > > > > > >> On Fri, 14 Nov 2014, Daniel O'Connor wrote: > > > > > >> > On 12 Nov 2014, at 19:43, Andreas Nilsson <andrnils@gmail.com> > > > > wrote: > > > > > >> > > unclear is the word for it :) And thanks for looking into > > this. > > > > > >> > > > > > >> ipmi/ilo is > > > > > >> > > > > > >> > > important on a server os. > > > > > >> > > > > > >> > > I found a reference to it in a ML post: > > > > > >> > > > > > > http://lists.freebsd.org/pipermail/freebsd-stable/2013-February/072464.ht > > > > > >> ml > > > > > >> > > > > > >> > I started that thread :) > > > > > >> > I did get it working on the hardware I was using (Supermicro > > X9SCL- > > F > > > > > >> > > > > > >> and X8SIL-F) > > > > > >> > > > > > >> > I used the following BIOS settings > > > > > >> > > > > > > >> > ? Remote Access - Enabled > > > > > >> > ? Serial Port Number - COM3 > > > > > >> > ? Serial Port Mode - 115200, 8, n, 1 > > > > > >> > ? Flow Control - Hardware > > > > > >> > ? Redirection After BIOS POST - Always > > > > > >> > ? Terminal Type - VT100 > > > > > >> > ? VT-UTF8 Combo Key Support - Disabled > > > > > >> > ? Sredir Memory Display Delay - No Delay > > > > > >> > > > > > > >> > And the following in loader.conf > > > > > >> > # Give preference to VGA console > > > > > >> > console="vidconsole,comconsole" > > > > > >> > # Uncomment below and comment above to give serial console > > > > preference > > > > > >> > #console="comconsole,vidconsole" > > > > > >> > comconsole_speed="115200" > > > > > >> > boot_multicons="YES" > > > > > >> > hint.uart.0.flags="0x0" > > > > > >> > hint.uart.2.at="isa" > > > > > >> > hint.uart.2.port="0x3E8" > > > > > >> > hint.uart.2.flags="0x30" > > > > > >> > > > > > > >> > And this in /etc/ttys > > > > > >> > # IPMI console > > > > > >> > # Note: The Java console viewer doesn't seem to be very smart > > as it > > > > > >> > > > > > >> doesn't > > > > > >> > > > > > >> > # properly support VT100 > > > > > >> > cuau2 "/usr/libexec/getty 3wire.115200" vt100 on > > secure > > > > > >> > > > > > > >> > I could then access it using ipmitool like so > > > > > >> > ipmitool -H remoteip -U ADMIN -I lanplus sol activate > > > > > >> > [login] > > > > > >> > export TERM=xterm > > > > > >> > > > > > > >> > Note that I wanted vidconsole by default because mostly the > > systems > > > > > >> > > > > > >> were used by people local to them, however we could break into the > > > > loader > > > > > >> and type 'set console=comconsole,vidconsole? and then get > > everything > > > > over > > > > > >> the serial console for remote trouble shooting. > > > > > >> > > > > > >> > You may also wish to check the IPMI configuration via the web > > > > interface > > > > > >> > > > > > >> - by default it will failover to port 0 and it has terrible > > default > > > > > >> passwords. I changed the passwords and forced it to use the > > dedicated > > > > > >> IPMI > > > > > >> port even if nothing was connected to it. > > > > > >> > > > > > >> Well, I'm almost done with most of our SM server, even > > concentrated > > > > > >> console on > > > > > >> our console server with such a simple config: > > > > > >> > > > > > >> ---- 8< ---- > > > > > >> # ipmi/sol console template > > > > > >> default ipmi { > > > > > >> > > > > > >> master localhost; > > > > > >> type exec; > > > > > >> exec /usr/local/bin/ipmitool -f > > /usr/local/etc/ipmi-pass - > > U > > > > > >> > > > > > >> root -I lanplus -H %.int sol activate; > > > > > >> > > > > > >> execsubst %=cs; > > > > > >> #idletimeout 6h; > > > > > >> > > > > > >> break 0 { string "~B"; } > > > > > >> > > > > > >> } > > > > > >> > > > > > >> console gwn1 { include ipmi; } > > > > > >> console gwn2 { include ipmi; } > > > > > >> console gwn3 { include ipmi; } > > > > > >> console gwn4 { include ipmi; } > > > > > >> console gwn5 { include ipmi; } > > > > > >> console gwn6 { include ipmi; } > > > > > >> console gwn7 { include ipmi; } > > > > > >> console gwn8 { include ipmi; } > > > > > >> > > > > > >> console gwc2 { include ipmi; } > > > > > >> ---- 8< ---- > > > > > >> > > > > > >> This has console logging (including possible panics) as a surplus > > > > > >> > > > > > >> -- > > > > > >> Sincerely, > > > > > >> D.Marck [DM5020, MCK-RIPE, > > > > DM3-RIPN] > > > > > >> [ FreeBSD committer: > > marck@FreeBSD.org > > > > ] > > > > > >> > > > > > > ------------------------------------------------------------------------ > > > > > >> *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- > > marck@rinet.ru > > > > *** > > > > > >> > > > > > > ------------------------------------------------------------------------ > > > > > > > > > > > > Hello again, > > > > > > > > > > > > Searching on hw.uart.console, I found: > > > > > > > > > > > > http://lists.freebsd.org/pipermail/svn-src-head/2013-February/044641.html > > > > > > , a very enlightening thread. > > > > > > > > > > > > Basically: "ohh, you want to use something other than COM1 and > > tried > > to > > > > > > get away with just changing hint.uart stuff, which has worked for a > > > > while, > > > > > > ha, no way..." No heads up, nothing. > > > > > > > > > > > > Sorry to say jhb@ but is not a rare case. It is if not the > > default, a > > > > > > very common setup on every HP server with iLO, and it holds for > > most > > > > all > > > > > > OOB style serial emulation I have ever had the (dis)pleasure of > > working > > > > > > with. > > > > > > > > This was done _specifically_ so you could use non-COM1 for both loader > > and > > > > kernel with one thing to change. That is, you don't use > > hint.uart.X.flags > > > > after this. I have used this with many SuperMicro servers that use > > COM2 > > > > and > > > > COM3 because I wanted the entire path (boot loader and kernel) to work, > > not > > > > the kernel only. Having only the kernel means I can't break into the > > > > loader > > > > prompt to boot a different kernel, single user, etc. > > > > > > > > To clarify, are you using _different_ serial ports for the loader vs > > the > > > > kernel? That is the use case I considered to be rare. Every single > > server > > > > I have ever worked with (though not iLO, mostly Dell and SuperMicro) > > uses > > > > the > > > > same COM port for serial redirection rather for SOL or via actual > > cables. > > > > I've yet to use a system that, for example, used COM1 for the loader > > and > > > > COM2 > > > > for the kernel. You are saying that every HP server uses COM1 for the > > > > loader > > > > and COM2 for the kernel? > > > > > > > > -- > > > > John Baldwin > > > > > > > > > > Hello, > > > > > > For Supermicro: > > > The box was in a rack when I started this, so I don't really know how the > > > bios is setup, but as far as I can gather, getting > > > "BTX loader yadda yadda" seems to always happen, on any COM-port. > > > The actual output from loading modules and so on only happens with > > specific > > > settings. Here ( as in an answer to Dmitry I accidentally sent in > > private ) > > > things seems to be on different COMs: Setting: comconsole_port=0x2F8 > > (com2) > > > show loader and goes dark on handoff to kernel > > > comconsole_port=0x3E8 (com3) show no loader and stays dark > > > > > > Neither of them gives me console even after successful boot. The only way > > > to get serial is setting hw.uart.console="br:9600" explicitly. > > > > On the SuperMicro systems I have played with (X8DTU and the X9DRW > > equivalent), > > comconsole_port=0x2f8 worked for both BIOS, loader, and kernel, but for > > BIOS > > to work I did have to enable Console Redirection in the BIOS. For 2U > > machines > > (X8DAH at least, I think also for X9 equivalent), SOL was on COM3 instead > > of > > COM2, but using comconsole_port alone worked for BIOS, loader, and kernel. > > > > I also set the speed to 115200 (conspeed) which IIRC might have been the > > default setting in the BIOS when Console Redirection was enabled via SOL. > > You can check that remotely for SOL via ipmitool by seeing what baud rate > > it is set to. I think if IPMI expects 115200 and you use 9600, that might > > explain black output. > > > > -- > > John Baldwin > > > > Hello, > > I spent the day with new ( as in directly out of the box ) supermicro > x9sc(a/i) machine, which as you say has SOL on com3. I don't have logs > available now, but short summary was that only way to get it work was to > set the hints.uart and then hw.uart.console="br:9600" ( or another speed ) > or some bogus value. > > I'll get back to you tomorrow with the specifics ( the files contains > kenv|grep uart , dmesg | grep uart , tail -4 /boot/loader.conf and results > with loader (shown|not shown), kernel output (shown|not shown) and login > prompt (shown|not shown). ipmitool sol info would also be good to have, e.g.: Set in progress : set-complete Enabled : true Force Encryption : true Force Authentication : true Privilege Level : USER Character Accumulate Level (ms) : 60 Character Send Threshold : 96 Retry Count : 7 Retry Interval (ms) : 1000 Volatile Bit Rate (kbps) : 115.2 Non-Volatile Bit Rate (kbps) : 115.2 Payload Channel : 1 (0x01) Payload Port : 623 is from an X8DTU system. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201411201351.29190.jhb>