Date: Thu, 20 Nov 2014 22:01:52 +0100 From: Andreas Nilsson <andrnils@gmail.com> To: John Baldwin <jhb@freebsd.org> Cc: Daniel O'Connor <doconnor@gsoft.com.au>, FreeBSD Stable Mailing List <freebsd-stable@freebsd.org>, Dmitry Morozovsky <marck@rinet.ru> Subject: Re: SuperMicro IPMI/SOL and ipmitool troubles Message-ID: <CAPS9%2BSv1dsxGfEs0KRgXnh3Xb3iG=-yZ5S9jkapgWNdkfPKh4Q@mail.gmail.com> In-Reply-To: <201411201351.29190.jhb@freebsd.org> References: <alpine.BSF.2.00.1411112022500.25949@woozle.rinet.ru> <201411201145.01162.jhb@freebsd.org> <CAPS9%2BSsU_Yimg6j980fB8by7HGRbBNxZ280yVArx4aV2nFynAQ@mail.gmail.com> <201411201351.29190.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Nov 20, 2014 at 7:51 PM, John Baldwin <jhb@freebsd.org> wrote: > On Thursday, November 20, 2014 1:09:06 pm Andreas Nilsson wrote: > > On Thu, Nov 20, 2014 at 5:45 PM, John Baldwin <jhb@freebsd.org> wrote: > > > > > On Wednesday, November 19, 2014 5:15:05 pm Andreas Nilsson wrote: > > > > On Wed, Nov 19, 2014 at 8:05 PM, John Baldwin <jhb@freebsd.org> > wrote: > > > > > > > > > On Wednesday, November 19, 2014 05:02:49 PM Andreas Nilsson wrote: > > > > > > On Wed, Nov 19, 2014 at 3:28 PM, Andreas Nilsson < > andrnils@gmail.com > > > > > > > > > wrote: > > > > > > > On Fri, Nov 14, 2014 at 7:30 PM, Dmitry Morozovsky < > marck@rinet.ru > > > > > > > > > wrote: > > > > > > >> Daniel, > > > > > > >> > > > > > > >> nice to see you here too ;) > > > > > > >> > > > > > > >> On Fri, 14 Nov 2014, Daniel O'Connor wrote: > > > > > > >> > On 12 Nov 2014, at 19:43, Andreas Nilsson < > andrnils@gmail.com> > > > > > wrote: > > > > > > >> > > unclear is the word for it :) And thanks for looking into > > > this. > > > > > > >> > > > > > > >> ipmi/ilo is > > > > > > >> > > > > > > >> > > important on a server os. > > > > > > >> > > > > > > >> > > I found a reference to it in a ML post: > > > > > > >> > > > > > > > > > http://lists.freebsd.org/pipermail/freebsd-stable/2013-February/072464.ht > > > > > > >> ml > > > > > > >> > > > > > > >> > I started that thread :) > > > > > > >> > I did get it working on the hardware I was using (Supermicro > > > X9SCL- > > > F > > > > > > >> > > > > > > >> and X8SIL-F) > > > > > > >> > > > > > > >> > I used the following BIOS settings > > > > > > >> > > > > > > > >> > ? Remote Access - Enabled > > > > > > >> > ? Serial Port Number - COM3 > > > > > > >> > ? Serial Port Mode - 115200, 8, n, 1 > > > > > > >> > ? Flow Control - Hardware > > > > > > >> > ? Redirection After BIOS POST - Always > > > > > > >> > ? Terminal Type - VT100 > > > > > > >> > ? VT-UTF8 Combo Key Support - Disabled > > > > > > >> > ? Sredir Memory Display Delay - No Delay > > > > > > >> > > > > > > > >> > And the following in loader.conf > > > > > > >> > # Give preference to VGA console > > > > > > >> > console="vidconsole,comconsole" > > > > > > >> > # Uncomment below and comment above to give serial console > > > > > preference > > > > > > >> > #console="comconsole,vidconsole" > > > > > > >> > comconsole_speed="115200" > > > > > > >> > boot_multicons="YES" > > > > > > >> > hint.uart.0.flags="0x0" > > > > > > >> > hint.uart.2.at="isa" > > > > > > >> > hint.uart.2.port="0x3E8" > > > > > > >> > hint.uart.2.flags="0x30" > > > > > > >> > > > > > > > >> > And this in /etc/ttys > > > > > > >> > # IPMI console > > > > > > >> > # Note: The Java console viewer doesn't seem to be very > smart > > > as it > > > > > > >> > > > > > > >> doesn't > > > > > > >> > > > > > > >> > # properly support VT100 > > > > > > >> > cuau2 "/usr/libexec/getty 3wire.115200" vt100 on > > > secure > > > > > > >> > > > > > > > >> > I could then access it using ipmitool like so > > > > > > >> > ipmitool -H remoteip -U ADMIN -I lanplus sol activate > > > > > > >> > [login] > > > > > > >> > export TERM=xterm > > > > > > >> > > > > > > > >> > Note that I wanted vidconsole by default because mostly the > > > systems > > > > > > >> > > > > > > >> were used by people local to them, however we could break > into the > > > > > loader > > > > > > >> and type 'set console=comconsole,vidconsole? and then get > > > everything > > > > > over > > > > > > >> the serial console for remote trouble shooting. > > > > > > >> > > > > > > >> > You may also wish to check the IPMI configuration via the > web > > > > > interface > > > > > > >> > > > > > > >> - by default it will failover to port 0 and it has terrible > > > default > > > > > > >> passwords. I changed the passwords and forced it to use the > > > dedicated > > > > > > >> IPMI > > > > > > >> port even if nothing was connected to it. > > > > > > >> > > > > > > >> Well, I'm almost done with most of our SM server, even > > > concentrated > > > > > > >> console on > > > > > > >> our console server with such a simple config: > > > > > > >> > > > > > > >> ---- 8< ---- > > > > > > >> # ipmi/sol console template > > > > > > >> default ipmi { > > > > > > >> > > > > > > >> master localhost; > > > > > > >> type exec; > > > > > > >> exec /usr/local/bin/ipmitool -f > > > /usr/local/etc/ipmi-pass - > > > U > > > > > > >> > > > > > > >> root -I lanplus -H %.int sol activate; > > > > > > >> > > > > > > >> execsubst %=cs; > > > > > > >> #idletimeout 6h; > > > > > > >> > > > > > > >> break 0 { string "~B"; } > > > > > > >> > > > > > > >> } > > > > > > >> > > > > > > >> console gwn1 { include ipmi; } > > > > > > >> console gwn2 { include ipmi; } > > > > > > >> console gwn3 { include ipmi; } > > > > > > >> console gwn4 { include ipmi; } > > > > > > >> console gwn5 { include ipmi; } > > > > > > >> console gwn6 { include ipmi; } > > > > > > >> console gwn7 { include ipmi; } > > > > > > >> console gwn8 { include ipmi; } > > > > > > >> > > > > > > >> console gwc2 { include ipmi; } > > > > > > >> ---- 8< ---- > > > > > > >> > > > > > > >> This has console logging (including possible panics) as a > surplus > > > > > > >> > > > > > > >> -- > > > > > > >> Sincerely, > > > > > > >> D.Marck [DM5020, MCK-RIPE, > > > > > DM3-RIPN] > > > > > > >> [ FreeBSD committer: > > > marck@FreeBSD.org > > > > > ] > > > > > > >> > > > > > > > > > ------------------------------------------------------------------------ > > > > > > >> *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- > > > marck@rinet.ru > > > > > *** > > > > > > >> > > > > > > > > > ------------------------------------------------------------------------ > > > > > > > > > > > > > > Hello again, > > > > > > > > > > > > > > Searching on hw.uart.console, I found: > > > > > > > > > > > > > > > > http://lists.freebsd.org/pipermail/svn-src-head/2013-February/044641.html > > > > > > > , a very enlightening thread. > > > > > > > > > > > > > > Basically: "ohh, you want to use something other than COM1 and > > > tried > > > to > > > > > > > get away with just changing hint.uart stuff, which has worked > for a > > > > > while, > > > > > > > ha, no way..." No heads up, nothing. > > > > > > > > > > > > > > Sorry to say jhb@ but is not a rare case. It is if not the > > > default, a > > > > > > > very common setup on every HP server with iLO, and it holds for > > > most > > > > > all > > > > > > > OOB style serial emulation I have ever had the (dis)pleasure of > > > working > > > > > > > with. > > > > > > > > > > This was done _specifically_ so you could use non-COM1 for both > loader > > > and > > > > > kernel with one thing to change. That is, you don't use > > > hint.uart.X.flags > > > > > after this. I have used this with many SuperMicro servers that use > > > COM2 > > > > > and > > > > > COM3 because I wanted the entire path (boot loader and kernel) to > work, > > > not > > > > > the kernel only. Having only the kernel means I can't break into > the > > > > > loader > > > > > prompt to boot a different kernel, single user, etc. > > > > > > > > > > To clarify, are you using _different_ serial ports for the loader > vs > > > the > > > > > kernel? That is the use case I considered to be rare. Every > single > > > server > > > > > I have ever worked with (though not iLO, mostly Dell and > SuperMicro) > > > uses > > > > > the > > > > > same COM port for serial redirection rather for SOL or via actual > > > cables. > > > > > I've yet to use a system that, for example, used COM1 for the > loader > > > and > > > > > COM2 > > > > > for the kernel. You are saying that every HP server uses COM1 for > the > > > > > loader > > > > > and COM2 for the kernel? > > > > > > > > > > -- > > > > > John Baldwin > > > > > > > > > > > > > Hello, > > > > > > > > For Supermicro: > > > > The box was in a rack when I started this, so I don't really know > how the > > > > bios is setup, but as far as I can gather, getting > > > > "BTX loader yadda yadda" seems to always happen, on any COM-port. > > > > The actual output from loading modules and so on only happens with > > > specific > > > > settings. Here ( as in an answer to Dmitry I accidentally sent in > > > private ) > > > > things seems to be on different COMs: Setting: comconsole_port=0x2F8 > > > (com2) > > > > show loader and goes dark on handoff to kernel > > > > comconsole_port=0x3E8 (com3) show no loader and stays dark > > > > > > > > Neither of them gives me console even after successful boot. The > only way > > > > to get serial is setting hw.uart.console="br:9600" explicitly. > > > > > > On the SuperMicro systems I have played with (X8DTU and the X9DRW > > > equivalent), > > > comconsole_port=0x2f8 worked for both BIOS, loader, and kernel, but for > > > BIOS > > > to work I did have to enable Console Redirection in the BIOS. For 2U > > > machines > > > (X8DAH at least, I think also for X9 equivalent), SOL was on COM3 > instead > > > of > > > COM2, but using comconsole_port alone worked for BIOS, loader, and > kernel. > > > > > > I also set the speed to 115200 (conspeed) which IIRC might have been > the > > > default setting in the BIOS when Console Redirection was enabled via > SOL. > > > You can check that remotely for SOL via ipmitool by seeing what baud > rate > > > it is set to. I think if IPMI expects 115200 and you use 9600, that > might > > > explain black output. > > > > > > -- > > > John Baldwin > > > > > > > Hello, > > > > I spent the day with new ( as in directly out of the box ) supermicro > > x9sc(a/i) machine, which as you say has SOL on com3. I don't have logs > > available now, but short summary was that only way to get it work was to > > set the hints.uart and then hw.uart.console="br:9600" ( or another speed > ) > > or some bogus value. > > > > I'll get back to you tomorrow with the specifics ( the files contains > > kenv|grep uart , dmesg | grep uart , tail -4 /boot/loader.conf and > results > > with loader (shown|not shown), kernel output (shown|not shown) and login > > prompt (shown|not shown). > > ipmitool sol info would also be good to have, e.g.: > > Set in progress : set-complete > Enabled : true > Force Encryption : true > Force Authentication : true > Privilege Level : USER > Character Accumulate Level (ms) : 60 > Character Send Threshold : 96 > Retry Count : 7 > Retry Interval (ms) : 1000 > Volatile Bit Rate (kbps) : 115.2 > Non-Volatile Bit Rate (kbps) : 115.2 > Payload Channel : 1 (0x01) > Payload Port : 623 > > is from an X8DTU system. > > -- > John Baldwin > Sure thing, below is the output from the machine where this started, for me at least. Still a X9SCI/X9SCA machine: Set in progress : set-complete Enabled : true Force Encryption : false Force Authentication : false Privilege Level : USER Character Accumulate Level (ms) : 0 Character Send Threshold : 0 Retry Count : 0 Retry Interval (ms) : 0 Volatile Bit Rate (kbps) : 115.2 Non-Volatile Bit Rate (kbps) : 115.2 Payload Channel : 1 (0x01) Payload Port : 623 Best regards Andreas
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPS9%2BSv1dsxGfEs0KRgXnh3Xb3iG=-yZ5S9jkapgWNdkfPKh4Q>