Hewlett Packard Enterprise M-Series User Manual

Download

Page 1

M-Series ONIE Switch User Manual

Rev 1.6 Mellanox ONIE Ver. 5.2.0004

Part number: P12821-001

Published: December 12, 2018

Edition: 1

Page 2

Notices

The information contained herein is subject to change without notice. The only warranties for Hewlett Packard En te rpri se pro du cts and ser vices are set for t h in the expres s warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein. Links to third-party websites take you outside the Hewlett Packard En ter prise w ebsi te. Hewlett Pa cka rd Enterp rise has no control over and is not responsible for information outside the Hewlett Packard Enterprise website. Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Compute r Sof tware, Comp ute r So ft ware Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Mellanox®, Mellanox logo, ConnectX®, MLNX-OS®, SwitchX®, Virtual Protocol Interconnect®, are registered tradema rks of Mell ano x Te ch nol ogi es, Ltd.

Al l othe r t rademarks are p rope rty of their resp ective owners .

For the most updated list of Mellanox trademarks, visit http://www.mellanox.com/page/trademarks

Page 3

Document Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . .

1.1 Accessing the Mellanox System for the First Time . . . . . . . . . . . . . . . . . . . 7

1.2 Log Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3 Booting Into ONIE System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

. . . . . . . . . . . . . . . . . . . . . 5

. . . . . . . . . . . . . . . . . . 7

Chapter 2 How-Tos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.1 Install Network OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.1.1 From DHCP Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.1.2 From TFTP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.1.3 From IPv6 Neighbors. . . . . . . . .

2.1.4 From Local USB Drive . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.1.5 Via SCP Local File . . . . . . . . . . . .

2.2 Boot Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2.1 Rescue Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2.2 Mellanox ONIE Update . . . . . . .

2.2.3 Uninstall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.2.4 Embed . . . . . . . . . . . . . . . . . . . .

2.2.5 Reinstall . . . . . . . . . . . . . . . . . . .

2.3 Update BIOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

. . . . . . . . . . . . . . . 11

. . . . . . . . . . . . . . . 13

. . . . . . . . . . . . . . . 16

Chapter 3 CLI Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1 ONIE Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Support and Other Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Customer Self Repair . . . . . . . . . . . . . . . . . . . . . . . .

Remote Support . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Websites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . 26

. . . . . . . . . . . . . . . . . . . . 26

. . . . . . . . . . . . . . . . . . . . 28

Page 4

Document Revision History

Table 1 - Document Revision History

Revision Date Description

Rev 1.6 December 07, 2018

Rev 1.5 January 25, 2018

Rev 1.4 May 31, 2016

Added Section 2.3, “Update BIOS,” on page 18 Enhanced user manual with printouts of ONIE Grub Menu Updated Section 2.2, “Boot Modes,” on page 14

Added:

• “onie-sysinfo -v” on page 24

Updated:

• “onie-sysinfo” on page 24

Updated:

• UM to reflect new software printouts

• Chapter 3, “CLI Reference” on page 19 with commands “onie-nos-install”, “onie-stop”, “onie-start”, and “onie-sysinfo”

Rev 1.3 January 15, 2015

Removed PPC content Updated:

• Section “Related Documentation” on page 5

• Section 2.1.2, “From TFTP Server,” on page 11

• Section 2.1.4, “From Local USB Drive,” on page 13

• Section 2.2.5, “Reinstall,” on page 18

• Section 2.2.3, “Uninstall,” on page 16

• Chapter 3, “CLI Reference” on page 19

Rev 1.2 August 31, 2014

Updated Section 2.1.4, “From Local USB Drive,” on page 13

Rev 1.1 July 2014 Removed sections 1.2, 1.3, 1.4 and 2.3.

Added Section 2.2.4, “Embed,” on page 17 Updated:

Table 2, “Reference Documents,” on page 5

•

• Chapter 3, “CLI Reference” on page 19

Rev 1.0 September 2013 First release

Page 5

Preface

Intended Audience

This document and the system is a bundle intended for system engineers and system developers interested in assembling their own OS on top of Mellanox switch systems. The system provides an easy way to install a user-defined OS over a Mellanox switch system.

This document contains instructions supplementary to the official O enced in the

“Related Documentation” section.

NIE documentation refer-

Documentation Conventions

Typographical Conventions

Table 3 - Typographical Conventions

Description Convention Example

File names file.extension

Directory names directory

Commands and their parameters command param1 mts3610-1 > show hosts

Required item < >

Optional item [ ]

Mutually exclusive parameters { p1, p2, p3 } or {p1 | p2 |

}

Opt

ional mutually exclusive

parameters

[ p1 | p2 | p3 ]

Page 6

Table 3 - Typographical Conventions

Description Convention Example

Comments to explain command examples

Variables for which users supply specific values

Emphasized words Italic font These are emphasized words

Note

Warni ng

// // This is a comment

Italic font enable

<text> This is a note.

<text> Make sure to connect to the

RS-232 RJ-45 port of the switch and no port.

Common Abbreviations and Acronyms

Table 4 - Abbreviations and Acronyms

Abbreviations Description

ONIE Open Network Install Environment

NOS Network OS

t to the MGT

Page 7

1 Introduction

Console

ONIE (Open Network Install Environment) is an open source initiative which enables automatic installation of user-defined OS whose function is to setup a network OS that provides the following services:

• Re/installing an OS

• Booting in "rescue" mode

• Formatting the system

For more information please refer to the following website:

https://github.com/opencomputeproject/onie.

1.1 Accessing the Mellanox System for the First Time

The system provided has DHCP enabled by default, therefore, login may be performed through the management ports (using "root" username and no password) and/or the serial console as explained in the following steps:

Introduction

Step 1. Connect the host PC to the CONSOLE (RJ-45) port of the switch system using the supplied

cable. The CONSOLE ports are shown below as examples.

Figure 1: Console Port

Make sure to connect to the CONSOLE port of the switch and not to the MGT port.

Page 8

Introduction

Step 2.

Configure a serial terminal program (for example, HyperTerminal, minicom, or Tera Term) on your host PC with the settings described in the table below.

Table 5 - Serial Terminal Program Configuration for 2016.11-5.2.0004-115200

Baud rate 115200

Data bits 8

Stop bits 1

Parity None

Flow Control None

Step 3. Activate the console by pressing Enter.

Please press Enter to activate this console. Info: eth0: Checking link... up. ** Installer Mode Enabled **

Step 4. (Optional) The system is configured to fetch an image using auto-discovery which is

enabled by default. To disable auto discovery, run:

ONIE:/ # onie-stop Stopping: discover... done.

1.2 Log Facilities

Parameter Setting

Mellanox ONIE uses the following log facilities:

• dmesg

• /var/log/messages

• /var/log/onie.log (Mellanox ONIE OS only)

• /var/log/syslog

The following is an example of

Nov 25 12:47:42 sys-hostname local0.info onie: Starting: klogd... Nov 25 12:47:42 sys-hostname local0.info onie: done. Nov 25 12:47:42 sys-hostname local0.info onie: Starting: dropbear ssh daemon... Nov 25 12:47:43 sys-hostname local0.info onie: done. Nov 25 12:47:43 sys-hostname local0.info onie: Starting: telnetd...

cat /var/log/onie.log:

Page 9

Nov 25 12:47:43 sys-hostname local0.info onie: done. Nov 25 12:47:43 sys-hostname local0.info onie: Starting: discover... Nov 25 12:47:43 sys-hostname local0.info onie: done. Nov 25 12:47:43 sys-hostname local0.info onie: Info: eth0: Checking link... Nov 25 12:47:43 sys-hostname local0.info onie: up. Nov 25 12:47:43 sys-hostname local0.info onie: Info: Trying DHCPv4 on interface: eth0 Nov 25 12:47:43 sys-hostname local0.info onie: ONIE: Using DHCPv4 addr: eth0: 10.7.144.133 / 255.255.248.0 Nov 25 12:47:43 sys-hostname local0.info discover: ONIE: Starting ONIE Service Discovery Nov 25 12:47:51 sys-hostname local0.info discover: [truncated] onie_neighs =

.144.1,10.7.150.251,10.7.150.252,[fe80::7efe:90ff:fe65:dca0-eth0],[fe80::e61d:2dff:fe5a:d72a-

0.250,10.7

10.7.15 eth0],[fe80::fe15:b4ff:fe97:224c-eth0],[fe80::2e0:86ff:fe29:5f74-eth0],[fe80::7efe:90ff: Nov 25 12:47:51 sys-hostname local0.info discover: [truncated] 61d:2dff:fef7:d984-eth0],[fe80::f652:14ff:fed5:b3c4-

fe80::268a

eth0],[ eth0],[fe80::f652:14ff:fefb:9294-eth0],[fe80::7efe:90ff:fec5:72 Nov 25 12:47:51 sys-hostname local0.info discover: [truncated] :2a8e-eth0],[fe80::202:c9ff:fe5e:da96eth0],[ eth0],[fe80::ee0d:9aff:fe9f:cade-eth0],[fe80::7210:6fff:feba:930c-eth0],[fe Nov 25 12:47:51 sys-hostname local0.info discover: ,[fe80::ee0d:9aff:feab:39bc-eth0],[fe80::da9d:67ff:fe65:e5e1eth0],[ eth0],[fe80::202:c9ff:fe63:e88e-eth0],[fe80::7efe:90ff:fe57:3e34Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_router = 10.7.144.1 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_subnet = 255.255.248.0 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_domain = mtl.labs.mlnx Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_interface = eth0 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_dns = 10.7.77.192 10.7.77.135 10.198.0.169 10.211.0.124 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_siaddr = 10.7.1.114 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_serverid = 10.7.77.135 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_boot_reason = install Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_broadcast = 10.7.151.255 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_boot_file = pxelinux.0 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_ip = 10.7.144.133 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_nmi_watchdog = 0 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_lease = 36000 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_ntpsrv = 10.7.77.134 10.7.77.135 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_mask = 21 Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_hostname = sys-hostname Nov 25 12:47:51 sys-hostname local0.info discover: onie_disco_opt53 = 05 Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-x86_64-mlnx_msn2100-r0 ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-x86_64-mlnx_msn2100-r0.bin ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-x86_64-mlnx_msn2100 ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-x86_64-mlnx_msn2100.bin ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-mlnx_msn2100 ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-mlnx_msn2100.bin ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-x86_64-mlnx ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-x86_64-mlnx.bin ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-x86_64 ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer-x86_64.bin ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.1.114/onie-installer.bin ...

:7ff:fe47:b480-eth0],[fe80::7210:6fff:febb:9966-eth0],[fe80::202:c9ff:fe63:242e-

fe80::7210

:6fff:febb:99ea-eth0],[fe80::7210:6fff:febb:995e-eth0],[fe80::e61d:2dff:fe6a:cb70-

fe80::7210

:6fff:feba:9364-eth0],[fe80::e61d:2dff:feff:3830-eth0],[fe80::202:c9ff:fe62:1fd8-

Introduction

Page 10

Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.77.135/onie-installer-x86_64-mlnx_msn2100-r0 ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.77.135/onie-installer-x86_64-mlnx_msn2100-r0.bin ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.77.135/onie-installer-x86_64-mlnx_msn2100 ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.77.135/onie-installer-x86_64-mlnx_msn2100.bin ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.77.135/onie-installer-mlnx_msn2100 ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.77.135/onie-installer-mlnx_msn2100.bin ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.77.135/onie-installer-x86_64-mlnx ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.77.135/onie-installer-x86_64-mlnx.bin ... Nov 25 12:48:00 sys-hostname local0.info onie-exec: Info: Attempting http://10.7.77.135/onie-installer-x86_64 ...

...

1.3 Booting Into ONIE System

When loading an ONIE system, the system loads the following GRUB menu:

GNU GRUB version 2.02

Introduction

To learn more about each of the boot mode options please refer to ONIE official documentation at the following URL:

https://opencomputeproject.github.io/onie/design-spec/x86_boot_loader.html?highlight=embed#onie-boot-commands-in-grub-prompt.

To uninstall or reistall the OS image, power cycle the switch and select the ONIE menu.

The user has 5 seconds to select the ONIE menu and must enter his attempts to initiate a network boot or NOS boot.

choice before ONIE

Page 11

2How-Tos

2.1 Install Network OS

The following subsections illustrate how to install a Network OS using different methods.

2.1.1 From DHCP Server

Perform the following in your DHCP server.

Step 1. Define a DHCP filename by configuring it in /etc/dhcpd.conf. Run:

filename "tftpboot_location/onie-installer-x86_64-mlnx_x86-r5.2.0004.bin";

Step 2. Enable auto-discovery mode (if disabled). Run:

ONIE:/ # onie-start Starting: discover... done. ONIE:/ #

Step 3. The switch system will fetch and install the OS image from the IP address discovered in the

previous step (e.g. 20.20.20.20—this address will be used throughout this manual).

Info: Fetching http://20.20.20.20/onie-installer-x86_64-mlnx_x86-r5.2.0004 ... ONIE: Executing installer: http://20.20.20.20/onie-installer-x86_64-mlnx_x86-r5.2.0004 Verifying image checksum ... OK. Preparing image archive ... OK. MLNX OS Installer: platform: x86_64-mlnx_x86-r5.0 ====== Invoking MLNX OS installer via kexec ====== Network will disconnect, further logs may be reviewed via serial console [1018004.495413] sd 0:0:0:0: [sda] Synchronizing SCSI cache [1018004.501473] Starting new kernel [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu ...

How-Tos

2.1.2 From TFTP Server

Perform the following in your TFTP server.

Step 1. Move the image file under the TFTP boot location.

Step 2. Enable auto-discovery mode (if disabled). Run:

ONIE:/ # onie-start Starting: discover... done.

Step 3. The switch system will fetch and install the OS image.

Info: Fetching tftp://20.20.20.20/onie-installer-x86_64-mlnx_x86-r5.2.0004 ... ONIE: Executing installer: tftp://20.20.20.20/onie-installer-x86_64-mlnx_x86-r5.2.0004 Verifying image checksum ... OK. Preparing image archive ... OK. MLNX OS Installer: platform: x86_64-mlnx_x86-r5.0 ====== Invoking MLNX OS installer via kexec ====== Network will disconnect, further logs may be reviewed via serial console

Page 12

[1018004.495413] sd 0:0:0:0: [sda] Synchronizing SCSI cache [1018004.501473] Starting new kernel [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu ...

2.1.3 From IPv6 Neighbors

 To install a Network OS from IPv6 neighbors using an installer test script:

Step 1. Boot ONIE in install mode by selecting "Install OS" in the ONIE menu.

GNU GRUB version 2.02

How-Tos

Step 2. Prepare a small installer test script at the deployment server (using HTTP server).

# cat /var/www/html/onie-isntaller-mlnx #/bin/sh

echo "Mellanox ONIE discovery over eth0 ipv6 neighbors test" exit 1

Step 3. Let the automatic installer detect the IPv6 neighbor you prepared.

ONIE: Using DHCPv6 addr: eth0: fe80::3652:b5fe:fecd:897d ONIE: Using DHCPv6 addr: eth1: fe80::202a:c9df:fe6e:68ab discover: installer mode detected. Running installer.

Please press Enter to activate this console. Info: Attempting http://[fe80::3640:b5ff:febf:897d%eth0]/onie-installer ... Info: Attempting http://[fe80::3640:b5ff:febf:897d%eth0]/onie-installer.bin ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-x86_64-mlnx_msn2700-r0 ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-x86_64-mlnx_msn2700-r0.bin ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-x86_64-mlnx_msn2700 ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-x86_64-mlnx_msn2700.bin ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-mlnx_msn2700 ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-mlnx_msn2700.bin ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-x86_64-mlnx ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-x86_64-mlnx.bin ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-x86_64 ... Info: Attempting http://[fe80::202:c9ff:fe63:68a0%eth0]/onie-installer-x86_64.bin ...

Page 13

2.1.4 From Local USB Drive

ONIE discovery and installation from USB does not support EXFAT (NTFS). Use

VFAT or FAT32 formatting instead.

 To install a Network OS from a USB drive:

Step 1. Put the NOS image inside the USB root directory.

Step 2. Mount and install the NOS version. Run:

ONIE:/ # mkdir /mnt/tmpusb ONIE:/ # mount /dev/sdb1 /mnt/tmpusb ONIE:/ # cd /mnt/tmpusb ONIE:/ # mnt/tmpusb # sync ONIE:/ # onie-nos-install /mnt/tmpusb/onie-installer-x86_64-mlnx_x86-r5.2.0004.bin

Rebooting into ONIE (re)install mode... ONIE:/ # umount: can't remount rootfs read-only The system is going down NOW! Sent SIGTERM to all processes Sent SIGKILL to all processes Restarting system. ...

How-Tos

2.1.5 Via SCP Local File

 To install a Network OS from a via SCP local file:

Step 1. Select "Install OS" in the ONIE GRUB menu.

Step 2. Disable DHCP auto-discovery (for non-DHCP configurations only). Run:

ONIE:/ # onie-stop discover: installer mode detected. Stopping: discover...start-stop-daemon: warning: killing process done.

Step 3. Assign static IP and netmask for eth0 (for non-DHCP configurations only):

ONIE:/ # ifconfig eth0 192.168.1.200 netmask 255.255.255.0

Ping SCP server to verify connection:

ONIE:/ # ping 192.168.1.21

Step 4. Create directory to which to copy NOS image. Run:

ONIE:/ # mkdir /tmp/nos

Page 14

How-Tos

Step 5.

Step 6. cd to directory and install NOS image:

Copy NOS image to /tmp/nos:

ONIE:/ # scp root@192.168.1.21:/tftpboot/cumulus-linux-3.5.3-mlx-amd64.bin /tmp/nos ... Do you want to continue connecting? (y/n) yes root@192.168.1.21’s password: ... cumulus-linux-3.5.3-mlx-amd64.bin 100% 212MB 19.3MB/s 00:11

ONIE:/ # cd /tmp/nos ONIE:/tmp/nos # onie-nos-install cumulus-linux-3.5.3-mlx-amd64.bin discover: installer mode detected. Stopping: discover... done. ONIE: Executing installer: cumulus-linux-3.5.3-mlx-amd64.bin Verifying image checksum ...OK. Preparing image archive ... OK. Verifying image architecture ...OK. Verifying system ram ...OK. Setting up installer ...

2.2 Boot Modes

When loading an ONIE system, the system loads the following GRUB menu:

GNU GRUB version 2.02

For a full description of the different boot mode options and their function, please refer to the official ONIE documentation referenced in the “Related Documentation” section.

2.2.1 Rescue Mode

 To boot in Rescue mode:

Page 15

How-Tos

Step 1.

Step 2. Run "rescue":

Select "Rescue" in the ONIE GRUB menu:

# rescue ONIE: Rescue Mode ... Version : master-201604201145 Build Date: 2016-04-20T13:08+0300 [ 5.798037] i8042: No controller found Info: Mounting kernel filesystems... done. Info: Mounting LABEL=ONIE-BOOT on /mnt/onie-boot ... ... discover: Rescue mode detected. Installer disabled.

GNU GRUB version 2.02

Please press Enter to activate this console. To check the install status inspect /var/log/onie.log. Try this: tail -f /var/log/onie.log

** Rescue Mode Enabled **

2.2.2 Mellanox ONIE Update

 To update the Mellanox ONIE:

Step 1. Select "Update ONIE" in the ONIE GRUB menu:

GNU GRUB version 2.02

Page 16

How-Tos

Step 2.

Run the command "onie-self-update":

ONIE:/ # onie-self-update http://20.20.20.20/onie-updater-x86_64-mlnx_x86-r5.2.0004 topping: discover... done. Info: Fetching http://20.20.20.20/onie-updater-x86_64-mlnx_x86-r5.2.0004 ... Connecting to 20.20.20.20 (20.20.20.20:80) installer 100% |*******************************| 24601k 0:00:00 ETA ONIE: Executing installer: http://<ip address>/onie-updater-x86_64-mlnx_x86-r5.2.0004 Verifying image checksum ... OK. Preparing image archive ... OK. ONIE: Version : master-201604201145 ONIE: Architecture : x86_64 ONIE: Machine : mlnx_x86 ONIE: Machine Rev : 5.2.0004 ONIE: Config Version: 1 Installing ONIE on: /dev/sda Rebooting... + killall -TERM dropbear + killall -TERM syslogd + [ -x /lib/onie/shutdown-machine ] + /lib/onie/s[1205783.482229] LPCI2C DBG: LPC Generic Decode Range 3 deconfigured: 0xfc2501 [1205783.490022] LPCI2C DBG: LPC Generic Decode Range 2 deconfigured: 0xfc2001 lpci2c 15706 0 [1205783.498523] LPCI2C DBG: lpci2c removed

- Live 0xffffffffa0000000 (O) ...

2.2.3 Uninstall

 To uninstall the OS from a system. Run:

Step 1. Select "Uninstall OS" in the ONIE GRUB menu:

GNU GRUB version 2.02

Page 17

How-Tos

Step 2.

2.2.4 Embed

 To boot in Embed mode:

Step 1. Select rescue mode in the ONIE GRUB menu:

Run the command "uninstall":

ONIE:/ # uninstall ONIE: OS Uninstall Mode ... Version : master-201604201145 Build Date: 2016-04-20T13:08+0300 [ 5.796828] i8042: No controller found Info: Mounting kernel filesystems... done. Info: Mounting LABEL=ONIE-BOOT on /mnt/onie-boot ... [ 10.270675] LPCI2C DBG: LPC Generic Decode Range 2 configured: 0xfc2001 [ 10.277368] LPCI2C DBG: LPC Generic Decode Range 3 configured: 0xfc2501 ... ONIE:/ # + /bin/umount -a -r The system is going down NOW! ...

GNU GRUB version 2.02

Step 2. Run the "onie-self-update" command.

ONIE:/ # onie-self-update -e http://20.20.20.20/sx_mlnx_os/onie/x86.r5.2.0004/on ie-updater-x86_64-mlnx_x86-r5.2.0004.bin Stopping: discover... done. Info: Fetching http://20.20.20.20/sx_mlnx_os/onie/x86.r5.2.0004/onie-updater-x86_64mlnx_x86-r5.2.0004 ... ... Installing ONIE on: /dev/sda Rebooting... + killall -TERM dropbear + killall -TERM syslogd + [ -x /lib/onie/shutdown-machine ] ... ONIE:/ # + /bin/umount -a -r The system is going down NOW! ... [ 2274.038521] reboot: machine restart ...

Page 18

2.2.5 Reinstall

There are two possible methods to reinstall the network OS on a x86 system:

1. Uninstall (Section 2.2.3 on page 16) and then install (Section 2.1 on page 11) the OS from ONIE instead (scrubs the former data).

2. Embed (Section 2.2.4 on page 17) and then install (Section 2.1 on page 11) the OS (does not scrub).

2.3 Update BIOS

This procedure is relevant to ONIE version 5.1.00.xx and higher.

 In order to update BIOS through ONIE:

1. Boot to ONIE update mode.

During a system boot in grub menu choose ONIE and after that sele option.

2. In boo

ted ONIE, start the stop discover process:

# onie-stop

How-Tos

ct "ONIE: Update ONIE"

3. BIOS release directory contains BIOS ROM file and its MD5sum.

4. Copy these files to any directory on the targ

# scp /auto/sw_system_release/sx_mlnx_bios/IvyBridge/0ABZS017_02.02.x01/ Release/0ABZS017.rom* root@10.209.48.47:/tmp

et system. For example:

5. Perform BIOS update. For example:

# bios_update.sh /tmp/0ABZS017.rom

Do not interrupt BIOS update process.

When calling the bios_update.sh script, the absolute path to BIOS image must be provided.

6. Powe

r cycle your system.

7. Check new BIOS version from ONIE update mode. Example:

# bios_version.sh BIOS Version: 4.6.5 BIOS Release Date: 15/04/2018 BIOS SubVersion: 0ABZS017_02.02.001

Page 19

3 CLI Reference

3.1 ONIE Commands

onie-syseeprom

Display the system EEPROM data block.

CLI Reference

Syntax Description

Default

History

Example

N/A

5.0.0100

ONIE:/ # onie-syseeprom TlvInfo Header: Id String: TlvInfo Version: 1 Total Length: 584 TLV Name Code Len Value

-------------------- ---- --- ----Product Name 0x21 64 MSN2700 Part Number 0x22 20 MSN2700-CS2F Serial Number 0x23 24 MT1607X11210 Base MAC Address 0x24 6 7C:FE:90:EB:BB:80 Manufacture Date 0x25 19 10/30/2013 10:03:00 Device Version 0x26 1 0 MAC Addresses 0x2A 2 128 Manufacturer 0x2B 8 Mellanox Vendor Extension 0xFD 36 0x00 0x00 0x81 0x19 0x00 0x16 0x01 0x01 0x03 0x6C 0x00 0x00 0x4D 0x4C 0x4E 0x58 0x02 0x01 0x0C 0x05 0x0E 0x02 0x10 0x06 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 Vendor Extension 0xFD 164 0x00 0x00 0x81 0x19 0x00 0x92 0x00 0x03 0x01 0xBF 0x00 0x00 0x4D 0x54 0x31 0x36 0x30 0x37 0x58 0x31 0x4E 0x32 0x37 0x30 0x30 0x2D 0x43 0x53 0x32 0x46 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x41 0x37 0x00 0x00 0x00 0xA1 0x8D 0xF6 0x50 0x61 0x6E 0x74 0x68 0x65 0x72 0x20 0x45 0x74 0x68 0x20 0x31 0x30 0x30 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x50 0x00 0x00 0x00 0x0A 0x8C 0x4D 0x53 0x4E 0x32 0x37 0x30 0x30 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 Vendor Extension 0xFD 36 0x00 0x00 0x81 0x19 0x00 0x10 0x00 0x03 0x05 0xE8 0x00 0x00 0x00 0x00 0x00 0x00

0x00 0x00 0x00 0

0x00 Vendor Extension 0xFD 36 0x00 0x00 0x81 0x19 0x00 0x1E 0x00 0x11 0x02 0xFF 0x00 0x00 0x0D 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0 Vendor Extension 0xFD 36 0x00 0x00 0x81 0x19 0x00 0x12 0x00 0x01 0x06 0x85 0x00 0x00 0x00 0x46 0x00 0x00 0x08 0x00 0x05 0x05 0 Platform Name 0x28 64 x86_64-mlnx_x86-r0 Loader Version 0x29 32 5.2.0004 CRC-32 0xFE 4 0x15860FD9 Checksum is valid. ONIE:/ #

0x31 0x32 0x31 0x30 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x4D 0x53

x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

x7C 0xFE 0x90 0xEB 0xBB 0x80 0x30 0x30 0x7C 0xFE 0x90 0x03 0x00 0xEB 0xBB 0x80

x05 0x05 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

Note

Page 20

onie-syseeprom --list

Lists the supported TLV (type, length and value) codes and names.

Syntax Description N/A

Default N/A

History 5.0.0100

CLI Reference

Example

Note

ONIE:~ # onie-syseeprom --list TLV Code TLV Name ======== ================= 0x21 Product Name 0x22 Part Number 0x23 0x24 Base MAC Address 0x25 Manufacture Date 0x26 Device Version 0x27 Label Revision 0x28 0x29 Loader Version 0x2a MAC Addresses 0x2b Manufacturer 0x2c Country Code 0x2d Vendor Name 0x2e Diag Version 0x2f Service Tag 0xfd Vendor Extension 0xfe CRC-32 ONIE:~ #

Serial Number

Platform Name

Page 21

onie-syseeprom --get

Locates a TLV by code and write the value to stdout.

Syntax Description N/A

Default N/A

History 5.0.0100

CLI Reference

Example

Note

ONIE:~ # onie-syseeprom --get 0x22 MSX1710-BS2F2E ONIE:~ #

Page 22

onie-nos-install

Installs a network operation system.

Syntax Description N/A

Default N/A

History 5.0.1404

CLI Reference

Example

Note

ONIE:~ # onie-nos-install http://<ip-address>/CumulusLinux-3.0.0-

1456539878.a71971a-amd64.bin Stopping: discover... done. Info: Fetching http://<ip-address>/CumulusLinux-3.0.0-

1456539878.a71971a-amd64.bin ... Connecting to <ip-address> (<ip-address>:<port>) installer 100% |*******************************| 271M 0:00:00 ETA ONIE: Executing installer: http://<ip-address>/CumulusLinux-3.0.0-

1456539878.a71971a-amd64.bin Verifying image checksum ... OK. Preparing image archive ... OK. Dumping image info... Control File Contents ===================== Description: Cumulus Linux OS-Release: 3.0.0~1456539878.a71971a-3d24735-201602270227-clbuilder Architecture: amd64 Date: Mon, 29 Feb 2016 13:15:17 -0800 Installer-Version: 1.2 Homepage: http://www.cumulusnetworks.com/ Data Archive Contents ...

Page 23

onie-stop

Disables DHCP auto-discovery.

Syntax Description N/A

Default N/A

History 5.0.1404

CLI Reference

Example

Note

onie-start

Syntax Description

Default

History

Example

Note

ONIE:~ # onie-stop Stopping: discover... done. ONIE:~ #

onie-start

Enables DHCP auto-discovery

N/A

5.0.1404

ONIE:~ # onie-start Starting: discover... done. ONIE:~ #

Page 24

onie-sysinfo

Displays platform information.

Syntax Description N/A

Default N/A

History 5.0.1404

5.1.0012 Updated example

CLI Reference

Example

Note

onie-sysinfo -v

Syntax Description

Default

History

Example

Note

ONIE:~ # onie-sysinfo x86_64-mlnx_msn2700-r0 ONIE:/ #

onie-sysinfo

Displays ONIE version details.

N/A

5.1.0012

ONIE:~ # onie-sysinfo -v

2016.11-5.2.0004-115200 ONIE:~ #

Page 25

Support and Other Resources

Accessing Hewlett Packard Enterprise Support

• For live assistance, go to the Contact Hewlett Packard Enterprise Worldwide website:

http://www.hpe.com/assistance

• To access documentation and support services, go to the Hewlett Packard Enterprise Support Center website:

http://www.hpe.com/support/hpesc

Information to Collect

• Technical support registration number (if applicable)

• Product name, model or version, and serial number

• Operating system name and version

• Firmware version

• Run each of the following in an ONIE session and

save their output:

1. dmidecode

2. onie-syseeprom

3. onie-syseeprom --list

• Extract the following files:

1. dmesg

2. /var/log/messages : tail -f /var/log/onie.log

3. /var/log/onie.log (Mellanox ONIE OS only)

4. /var/log/syslog

• Error messages

• Product-specific reports and logs

• Add-on products or components

• Third-party products or components

Page 26

Accessing Updates

• Some software products provide a mechanism for accessing software updates through the product interface. Review your product documentation to identify the recommende d software update method.

• To download product updates:

Hewlett Packard Enterprise Support Center

wwww.hpe.com/support/hpesc

Hewlett Packard Enterprise Support Center: Software downloads

wwww.hpe.com/support/downloads

Software Depot

wwww.hpe.com/support/softwaredepot

• To subscribe to eNewsletters and alerts:

wwww.hpe.com/support/e-updates

• To view and update your entitlements, and to link your contracts and warranties with your profile, go to the Hewlett Packard Enterprise Support Center More Information o n Access to Support Materials page:

www.hpe.com/support/AccessToSupportMaterials

Important: Access to some updates might require product entitlement when accessed

through the Hewlett Packard Enterprise Support Center. You must have an HPE Passport set up with relevant entitlements.

Customer Self Repair

Hewlett Packard En uct. If a CSR part needs to be replaced, it will be shipped directly to you at your convenience. Some parts do not qualify for CSR. Your Hewlett Packard Enterprise authorized service provider will determine whether a repair can be ac

For more information about CSR, contact your local service provider or go to the CSR website:

http://www.hpe.com/support/selfrepair

terprise customer self repair (CSR) programs al

low you to repair your prod-

so that you can install it

complished by CSR.

Page 27

Remote Support

Remote port agreement. It provides intelligent event diagnosis, and automatic, secure submission of hard-ware event notifications to Hewlett Packard Enterprise, which will initiate a fast and accurate resolution based on your product's service level. Hewlett Packard Enterprise strongly recom-mends that you register your device for remote support.

If your product includes additional remote support details, use search to locate that information.

support is available with supported devices as part of your warranty or contractual sup-

Remote Support and Proactive Care Information

HPE Get Connected

www.hpe.com/services/getconnected

HPE Proactive Care services

www.hpe.com/services/proactivecare

HPE Proactive Care service: Supported products list

www.hpe.com/services/proactivecaresupportedproducts

HPE Proactive Care advanced service: Supported products list

www.hpe.com/services/proactivecareadvancedsupportedproducts

Proactive Care Customer Information

Proactive Care central

www.hpe.com/services/proactivecarecentral

Proactive Care service activation

www.hpe.com/services/proactivecarecentralgetstarted

Warranty Information

To view the warranty for your product or to view the Safety and Compliance Information for Server, Storage, Power, Networking, and Rack Products reference document, go to the Enterprise Safety and Compliance website:

www.hpe.com/support/Safety-Compliance-EnterpriseProducts

Additional warranty information

HPE ProLiant and x86 Servers and Options

www.hpe.com/support/ProLiantServers-Warranties

HPE Enterprise Servers

www.hpe.com/support/EnterpriseServers-Warranties

HPE Storage Products

www.hpe.com/support/Storage-Warranties

HPE Networking Products

www.hpe.com/support/Networking-Warranties

Page 28

Regulatory Information

To view the regulatory information for your product, view the Safety and Compliance Information for Server, Storage, Power, Networking, and Rack Products, available at the Hewlett Packard Enterprise Support Center:

www.hpe.com/support/Safety-Compliance-EnterpriseProducts

Additional Regulatory Information

Hewlett Packard Enterprise is committed to providing our customers with information about the chemical substances in our products as needed to comply with legal requirements such a s REACH (Regulation EC No 1907/2006 of the European Parliament and the Council). A chemical information report for this product can be found at:

www.hpe.com/info/reach

For Hewlett Packard Enterprise product environmental and safety information and complianc e data, including RoHS and REACH, see:

www.hpe.com/info/ecodata

For Hewlett Packard Enterprise environmental information, including company programs, product recycling, and energy efficiency, see:

www.hpe.com/info/environment

Documentation Feedback

Hewlett Packard Enterprise is committed to providing documentation that meets your needs. To help us improve the documentation, send any errors, suggestions, or comments to Documentation Feedback ( title, part number, edition, and publication date located on the front cover of the document. For online help content, include the product name, product version, help edition, and publication date located on the legal notices page.

docsfeedback@hpe.com). When submitting your feedback, include the document

Websites

General Websites

Hewlett Packard Enterprise Information Library

www.hpe.com/info/EIL

Single Point of Connectivity Knowledge (SPOCK) Storage compatibility matrix

www.hpe.com/storage/spock

Storage white papers and analyst reports

www.hpe.com/storage/whitepapers

For additional websites, see Support and other resources.

Hewlett Packard Enterprise M-Series User Manual

Table of Contents

Document Revision History

Preface

Intended Audience

Related Documentation

Documentation Conventions

Common Abbreviations and Acronyms

1 Introduction

1.1 Accessing the Mellanox System for the First Time

1.2 Log Facilities

1.3 Booting Into ONIE System

2How-Tos

2.1 Install Network OS

2.1.1 From DHCP Server

2.1.2 From TFTP Server

2.1.3 From IPv6 Neighbors

2.1.4 From Local USB Drive

2.1.5 Via SCP Local File

2.2 Boot Modes

2.2.1 Rescue Mode

2.2.2 Mellanox ONIE Update

2.2.3 Uninstall

2.2.4 Embed

2.2.5 Reinstall

2.3 Update BIOS

3 CLI Reference

3.1 ONIE Commands

onie-syseeprom

onie-syseeprom --list

onie-syseeprom --get

onie-nos-install

onie-stop

onie-start

onie-sysinfo

onie-sysinfo -v

Support and Other Resources

Accessing Hewlett Packard Enterprise Support

Customer Self Repair

Warranty Information

Documentation Feedback

Websites

General Websites