IPMI on Penguin Computing Relion 1600

It is fairly easy to set up IPMI on the Penguin Computing Relion 1600: Everything from basic management (power off, power on, chassis status) to Serial Over LAN. The entire process shouldn't take more than 30 minutes.

One important thing to note: You cannot connect to the BMC over eth0 or eth1. The Relion 1600 has a separate "Management Port" for the BMC, which requires a separate ethernet cable and a unique IP address. So, before getting started, run one cable to eth0 and one cable to the Management Port, and have the IP address that you plan to use for the Management Port ready.

Install ipmitool

  1. Download the latest version of ipmitool from the ipmitool website: http://ipmitool.sourceforge.net/
  2. Do the standard configure; make; make install on the Relion. Make sure ipmitool compiles with the lanplus option enabled for IPMIv2 support (as long as it finds the openssl libraries this will happen).
  3. Modprobe the ipmi modules if they are not compiled into the kernel:
    modprobe ipmi_si modprobe ipmi_devintf
    If these commands return errors, go to the next step to see if the modules were perhaps compiled into the kernel.
  4. Test to see if you actually have an IPMI-enabled kernel:
    grep ipmidev /proc/devices

    If you don't get any response from that, then you don't have an IPMI-enabled kernel. Stop now and compile a kernel with IPMI support by enabling the following kernel config options:

    Device Drivers --> Character Devices --> IPMI --> [*] IPMI top-level message handler [*] Device interface for IPMI [*] IPMI System Interface handler [*] IPMI Watchdog Timer
  5. Assuming you have an IPMI-enabled kernel, check to ensure /dev/ipmi0 exists. If it does not, you can create it manually like this:
    mknod /dev/ipmi0 c `grep ipmidev /proc/devices | awk '{print $1}'` 0
  6. Test it and see the default settings:
    [root@computer ~]# ipmitool lan print 2 Set in Progress : Set Complete Auth Type Support : NONE MD5 PASSWORD Auth Type Enable : Callback : NONE MD5 PASSWORD : User : NONE MD5 PASSWORD : Operator : NONE MD5 PASSWORD : Admin : NONE MD5 PASSWORD : OEM : NONE MD5 PASSWORD IP Address Source : Static Address IP Address : 192.168.2.100 Subnet Mask : 255.255.255.0 MAC Address : 00:a0:d1:e3:07:d5 SNMP Community String : public IP Header : TTL=0x40 Flags=0x40 Precedence=0x00 TOS=0x10 Default Gateway IP : 192.168.2.203 RMCP+ Cipher Suites : Cipher Suite Priv Max : Not Available
  7. Set the IP address and gateway for the BMC. Do not use the same IP address that you used for eth0 or eth1; instead, give the BMC its own IP address, and set the gateway to match the rest of your network:
    ipmitool lan set 2 ipaddr XXX.XXX.XXX.XXX ipmitool lan set 2 defgw ipaddr XXX.XXX.XXX.XXX
  8. Set the channel password:
    ipmitool lan set 2 password ******** (type your password)
  9. Grant the default user Administrative privileges:
    ipmitool channel setaccess 2 1 ipmi=on privilege=4
  10. Make sure you have an ethernet cable running to the Management Port. First test ipmitool from the local machine:
    ipmitool chassis status
    Then test it from a remote machine:
    ipmitool -I lanplus -H XXX.XXX.XXX.XXX chassis status
    These commands should return the same results.

Serial Over LAN

Note on baud rate: After a bit of experimentation I eventually settled on an SOL baud rate of 38400. The higher you set this, the faster the connection will be, but in my experience anything higher than 38400 resulted in display problems (dropped characters) on the SOL terminal. Safe baud rates in Linux are 9600, 19200, 38400, 57600, and 115200. Experiment to find the one that works best for you, but remember that you must set the baud rate in the BIOS to be as high as or higher than what you set in Linux.

  1. In the BIOS, set the following:
  2. Make sure /dev/tty0 exists. If you have /dev/vc/0 but not /dev/tty0, then symlink /dev/tty0 -> vc/0.
    [[ ! -e /dev/tty0 ]] && [[ -e /dev/vc/0 ]] && ln -s vc/0 /dev/tty0

    Note: tty0 is just an alias for the current virtual terminal and it is where messages from the system are sent. Messages from the system will be seen on the console regardless of which virtual terminal it is displaying.

  3. Make sure /dev/ttyS0 exists. If not, create it:
    mknod -m 660 /dev/ttyS0 c 4 64
  4. If /etc/securetty exists and it does not already contain ttyS0, then we need to add ttyS0 to /etc/securetty:
    [[ -f /etc/securetty ]] && $( ! grep -q ^ttyS0 /etc/securetty ) && echo ttyS0 >> /etc/securetty
  5. Add the following line to the "Run gettys" section of /etc/inittab:
    sol:2345:respawn:/sbin/agetty ttyS0 38400

  6. If you are using GRUB:

    1. Add these lines to /etc/grub.conf:
      serial --unit=0 ---speed=38400 terminal --timeout=0 serial
    2. Append the following to any "kernel" lines in /etc/grub.conf:
      3. console=tty0 console=ttyS0,38400n8
    3. Comment out any lines mentioning a splash image.

    If you are using LILO:

    1. Add the following 'append' line to /etc/lilo.conf
      append="console=tty0 console=ttyS0,38400n8"
    2. Comment out any lines mentioning a splash image.

    Note: The order of the "console=" lines above is important. Whichever one comes LAST will be the one where the init sequence messages are sent. If you are managing your server using SOL, you probably want the init sequence messages to be sent to the SOL, but if you start working at the console instead, you should change this so console=tty0 comes last.

    Note #2 If you see any lines about a "splash image" in either grub.conf or lilo.conf, you need to comment them out. Splash images will not work right with a SOL connection.

  7. Make a connection from a remote machine:
    1. Start with an xterm set to the standard console size of 80x25. You can resize it later, but if you plan to do anything in the BIOS, the screen may do odd things if it is not set to 80x25.
    2. Run this command with your IPMI IP address:
      ipmitool -I lanplus -H XXX.XXX.XXX.XXX sol activate
    3. In a second xterm run the ipmitool chassis off/on and you'll see the console output on the sol console.
    4. BIOS TIP: To get into the BIOS using SOL, you must type Esc-4 instead of F4 (which is what the screen tells you to press). The same goes for any other F-key. Use Esc-2 instead of F2, etc.
    5. Once you are logged in you can resize your terminal, but after you do this, things like editors (emacs) and pagers (less) will not format properly until you run the following stty commands:
      stty rows ROWS stty cols COLS

      (Replace "ROWS" and "COLS" with the actual number of rows and columns in your terminal window.)

Additional Resources