mirror of
https://github.com/xcat2/xcat-core.git
synced 2025-05-29 09:13:08 +00:00
Refactor the mellanox installation instructions to make it easier to follow
This commit is contained in:
Victor Hu
parent
ffba03e3bc
commit
e10b5afad3
@ -0,0 +1,11 @@
|
||||
Configuration
|
||||
=============
|
||||
|
||||
The process to configure the osimage to install the Mellanox OFED Drivers for Diskful and Diskless scenarios are outlined below.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
mlnxofed_ib_install_v2_diskful.rst
|
||||
mlnxofed_ib_install_v2_diskless.rst
|
||||
|
||||
@ -7,8 +7,7 @@ Mellanox provides a tested and packaged version of the OpenFabrics Enterprise Di
|
||||
:maxdepth: 2
|
||||
|
||||
mlnxofed_ib_install_v2_preparation.rst
|
||||
mlnxofed_ib_install_v2_diskful.rst
|
||||
mlnxofed_ib_install_v2_diskless.rst
|
||||
mlnxofed_configuration.rst
|
||||
mlnxofed_ib_verified_scenario_matrix.rst
|
||||
mlnxofed_ib_known_issue.rst
|
||||
|
||||
|
||||
@ -1,169 +1,38 @@
|
||||
Configuration for Diskful Installation
|
||||
=======================================
|
||||
|
||||
1. Set script ``mlnxofed_ib_install`` as ``postbootscripts`` or ``postscripts`` ::
|
||||
|
||||
chdef <node> -p postbootscripts="mlnxofed_ib_install -p /install/<subpath>/<MLNX_OFED_LINUX.iso>"
|
||||
|
||||
Or ::
|
||||
|
||||
chdef <node> -p postscripts="mlnxofed_ib_install -p /install/<subpath>/<MLNX_OFED_LINUX.iso>"
|
||||
|
||||
xCAT simulates completely the way Mellanox scripts work by using ``postbootscripts``. This way need to reboot after drive installation to make Mellanox drivers work reliably just like Mellanox suggested. If you want to use the reboot after operating system installation to avoid reboot twice, you can using ``postscripts`` attribute to install Mellanox drivers. This way has been verified in limited scenarios. For more information please refer to :doc:`The Scenarios Have Been Verified </advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix>`. You can try this way in other else scenarios if you needed.
|
||||
|
||||
2. Specify dependency package
|
||||
|
||||
Some dependencies need to be installed before running Mellanox scripts. These dependencies are different between different scenario. xCAT configurates these dependency packages by using ``pkglist`` attribute of ``osimage`` definition. Please refer to :doc:`Add Additional Software Packages </guides/admin-guides/manage_clusters/ppc64le/diskful/customize_image/additional_pkg>` for more information::
|
||||
|
||||
# lsdef -t osimage <os>-<arch>-install-compute
|
||||
Object name: <os>-<arch>-install-compute
|
||||
imagetype=linux
|
||||
....
|
||||
pkgdir=/<os packages directory>
|
||||
pkglist=/<os packages list directory>/compute.<os>.<arch>.pkglist
|
||||
....
|
||||
|
||||
You can append the ib dependency packages list in the end of ``/<os packages list directory>/compute.<os>.<arch>.pkglist`` directly like below: ::
|
||||
|
||||
#cat /<os packages list directory>/compute.<os>.<arch>.pkglist
|
||||
@base
|
||||
@x11
|
||||
openssl
|
||||
ntp
|
||||
rsyn
|
||||
#ib part
|
||||
createrepo
|
||||
kernel-devel
|
||||
kernel-source
|
||||
....
|
||||
|
||||
|
||||
Or if you want to isolate InfiniBand dependency packages list into a separate file, after you edit this file, you can append the file in ``/<os packages list directory>/compute.<os>.<arch>.pkglist`` like below way: ::
|
||||
|
||||
#cat /<os packages list directory>/compute.<os>.<arch>.pkglist
|
||||
@base
|
||||
@x11
|
||||
openssl
|
||||
ntp
|
||||
rsyn
|
||||
#INCLUDE:/<ib pkglist path>/<you ib pkglist file>#
|
||||
|
||||
xCAT has shipped some ib pkglist files under ``/opt/xcat/share/xcat/ib/netboot/<ostype>/``, these pkglist files have been verified in sepecific scenario. Please refer to :doc:`The Scenarios Have Been Verified </advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix>` to judge if you can use it directly in your environment. If so, you can use it like below: ::
|
||||
|
||||
#cat /<os packages list directory>/compute.<os>.<arch>.pkglist
|
||||
@base
|
||||
@x11
|
||||
openssl
|
||||
ntp
|
||||
rsyn
|
||||
#INCLUDE:/opt/xcat/share/xcat/ib/netboot/<ostype>/ib.<os>.<arch>.pkglist#
|
||||
|
||||
Take rhels7.2 on ppc64le for example: ::
|
||||
|
||||
# lsdef -t osimage rhels7.2-ppc64le-install-compute
|
||||
Object name: rhels7.2-ppc64le-install-compute
|
||||
imagetype=linux
|
||||
osarch=ppc64le
|
||||
osdistroname=rhels7.2-ppc64le
|
||||
osname=Linux
|
||||
osvers=rhels7.2
|
||||
otherpkgdir=/install/post/otherpkgs/rhels7.2/ppc64le
|
||||
pkgdir=/install/rhels7.2/ppc64le
|
||||
pkglist=/install/custom/install/rh/compute.rhels7.ib.pkglist
|
||||
profile=compute
|
||||
provmethod=install
|
||||
template=/opt/xcat/share/xcat/install/rh/compute.rhels7.tmpl
|
||||
|
||||
|
||||
**[Note]**: If the osimage definition was generated by xCAT command ``copycds``, default value ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` was assigned to ``pkglist`` attribute. ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` is the sample pkglist shipped by xCAT, recommend to make a copy of this sample and using the copy in real environment. In the above example, ``/install/custom/install/rh/compute.rhels7.ib.pkglist`` is a copy of ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist``. ::
|
||||
|
||||
# cat /install/custom/install/rh/compute.rhels7.ib.pkglist
|
||||
#Please make sure there is a space between @ and group name
|
||||
wget
|
||||
ntp
|
||||
nfs-utils
|
||||
net-snmp
|
||||
rsync
|
||||
yp-tools
|
||||
openssh-server
|
||||
util-linux
|
||||
net-tools
|
||||
#INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist#
|
||||
|
||||
|
||||
|
||||
3. Install node ::
|
||||
|
||||
nodeset <node> osimage=<osver>-<arch>-install-compute
|
||||
rsetboot <node> net
|
||||
rpower <node> reset
|
||||
|
||||
|
||||
After steps above, you can login target node and find the Mellanox InfiniBand drives are located under ``/lib/modules/<kernel_version>/extra/``.
|
||||
|
||||
Issue ``ibv_devinfo`` command you can get the InfiniBand apater information ::
|
||||
|
||||
# ibv_devinfo
|
||||
hca_id: mlx5_0
|
||||
transport: InfiniBand (0)
|
||||
fw_ver: 10.14.2036
|
||||
node_guid: f452:1403:0076:10e0
|
||||
sys_image_guid: f452:1403:0076:10e0
|
||||
vendor_id: 0x02c9
|
||||
vendor_part_id: 4113
|
||||
hw_ver: 0x0
|
||||
board_id: IBM1210111019
|
||||
phys_port_cnt: 2
|
||||
Device ports:
|
||||
port: 1
|
||||
state: PORT_INIT (2)
|
||||
max_mtu: 4096 (5)
|
||||
active_mtu: 4096 (5)
|
||||
sm_lid: 0
|
||||
port_lid: 65535
|
||||
port_lmc: 0x00
|
||||
link_layer: InfiniBand
|
||||
|
||||
port: 2
|
||||
state: PORT_DOWN (1)
|
||||
max_mtu: 4096 (5)
|
||||
active_mtu: 4096 (5)
|
||||
sm_lid: 0
|
||||
port_lid: 65535
|
||||
port_lmc: 0x00
|
||||
link_layer: InfiniBand
|
||||
|
||||
Using ``service openibd status`` to verify if openibd works well. Below is the output in rhels7.2. ::
|
||||
|
||||
|
||||
# service openibd status
|
||||
HCA driver loaded
|
||||
|
||||
Configured IPoIB devices:
|
||||
ib0 ib1
|
||||
|
||||
Currently active IPoIB devices:
|
||||
Configured Mellanox EN devices:
|
||||
|
||||
Currently active Mellanox devices:
|
||||
|
||||
The following OFED modules are loaded:
|
||||
|
||||
rdma_ucm
|
||||
rdma_cm
|
||||
ib_addr
|
||||
ib_ipoib
|
||||
mlx4_core
|
||||
mlx4_ib
|
||||
mlx4_en
|
||||
mlx5_core
|
||||
mlx5_ib
|
||||
ib_uverbs
|
||||
ib_umad
|
||||
ib_ucm
|
||||
ib_sa
|
||||
ib_cm
|
||||
ib_mad
|
||||
ib_core
|
||||
|
||||
|
||||
Diskful Installation
|
||||
====================
|
||||
|
||||
#. Prepare dependency packages in the pkglist
|
||||
|
||||
In order for the Mellanox installation script to execute successfully, certain dependency packages are required to be installed on the compute node. xCAT provides sample package list files to help resolve these dependencies. The samples are located at ``/opt/xcat/share/xcat/ib/netboot/<os>/``.
|
||||
|
||||
To use the ``/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist``, edit your existing ``pkglist`` file for the target osimage and add the following at the bottom: ::
|
||||
|
||||
#INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist#
|
||||
|
||||
#. Configure the ``mlnxofed_ib_install`` script to install the MNLX_OFED drivers
|
||||
|
||||
xCAT has a concept of postscripts that can be used to customize the node after the operating system is installed.
|
||||
|
||||
Mellanox recommends that the operating system is rebooted after the drivers are installed, so xCAT recommends using the ``postscripts`` attribute to avoid the need for a second reboot. To invoke the ``mlnxofed_ib_install`` as a postscript ::
|
||||
|
||||
chdef -t node -o <node_name> \
|
||||
-p postscripts="mlnxofed_ib_install -p /install/<path-to>/<MLNX_OFED_LINUX.iso>"
|
||||
|
||||
#. Provision the node
|
||||
|
||||
#. Verification
|
||||
|
||||
* The Mellanox IB drivers are located at: ``/lib/modules/<kernel_version>/extra/``
|
||||
|
||||
* Use the ``ibv_devinfo`` comamnd to obtain information about the InfiniBand adapter
|
||||
|
||||
* Check the status of ``openibd`` service
|
||||
|
||||
sysVinit: ::
|
||||
|
||||
service openibd status
|
||||
|
||||
systemd: ::
|
||||
|
||||
systemctl status openibd.service
|
||||
|
||||
|
||||
@ -1,169 +1,58 @@
|
||||
Configuration for Diskless Installation
|
||||
=======================================
|
||||
|
||||
1. Specify dependency package
|
||||
|
||||
Some dependencies need to be installed before running Mellanox scripts. These dependencies are different among different scenarios. xCAT can help user to install these dependency packages by adding these package names to the file specified by the ``pkglist`` attribute of the ``osimage`` definition. Please refer to :doc:`Add Additional Software Packages </guides/admin-guides/manage_clusters/ppc64le/diskful/customize_image/additional_pkg>` for more information::
|
||||
|
||||
# lsdef -t osimage <osver>-<arch>-netboot-compute
|
||||
Object name: <osver>-<arch>-netboot-compute
|
||||
imagetype=linux
|
||||
....
|
||||
pkgdir=/<os packages directory>
|
||||
pkglist=/<os packages list directory>/compute.<os>.<arch>.pkglist
|
||||
....
|
||||
|
||||
You can append the ib dependency packages list in the end of ``/<os packages list directory>/compute.<os>.<arch>.pkglist`` directly like below: ::
|
||||
|
||||
#cat /<os packages list directory>/compute.<os>.<arch>.pkglist
|
||||
bash
|
||||
nfs-utils
|
||||
openssl
|
||||
dhclient
|
||||
kernel
|
||||
.....
|
||||
|
||||
#ib part
|
||||
createrepo
|
||||
kernel-devel
|
||||
kernel-source
|
||||
....
|
||||
|
||||
|
||||
Or if you want to isolate InfiniBand dependency packages list into a separate file, after you edit this file, you can append the file in ``/<os packages list directory>/compute.<os>.<arch>.pkglist`` like below way: ::
|
||||
|
||||
#cat /<os packages list directory>/compute.<os>.<arch>.pkglist
|
||||
bash
|
||||
nfs-utils
|
||||
openssl
|
||||
dhclient
|
||||
kernel
|
||||
.....
|
||||
#INCLUDE:/<ib pkglist path>/<you ib pkglist file>#
|
||||
|
||||
xCAT ships some InfiniBand pkglist files under ``/opt/xcat/share/xcat/ib/netboot/<ostype>/``, these pkglist files have been verified in sepecific scenario. Please refer to :doc:`The Scenarios Have Been Verified </advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix>` to judge if you can use it directly in your environment. If so, you can use it like below: ::
|
||||
|
||||
#cat /<os packages list directory>/compute.<os>.<arch>.pkglist
|
||||
bash
|
||||
nfs-utils
|
||||
openssl
|
||||
dhclient
|
||||
kernel
|
||||
.....
|
||||
#INCLUDE:/opt/xcat/share/xcat/ib/netboot/<ostype>/ib.<os>.<arch>.pkglist#
|
||||
|
||||
|
||||
2. Prepare postinstall scripts
|
||||
|
||||
Edit ``postinstall`` script to trigger InfniBand drvices installation during ``genimage``. Using below command to find out where the ``postinstall`` script is defined. ::
|
||||
|
||||
# lsdef -t osimage <os>-<arch>-netboot-compute
|
||||
Object name: <os>-<arch>-netboot-compute
|
||||
....
|
||||
postinstall=/<postinstall script path/compute.<os>.<arch>.postinstall
|
||||
....
|
||||
|
||||
|
||||
|
||||
Edit ``/<postinstall script path/compute.<os>.<arch>.postinstall`` and add below line in the end. ::
|
||||
|
||||
/install/postscripts/mlnxofed_ib_install \
|
||||
-p /install/<path>/<MLNX_OFED_LINUX.iso> -i $1 -n genimage
|
||||
|
||||
|
||||
**[Note]** Mellanox OFED ISO was built against a series of certain kernael versions, If the version of linux kernel you are using does not match with any of the Mellanox offered pre-built kernel modules, you can pass ``--add-kernel-support`` command line argument to Mellanox OFED installation script to build these kernel modules base on the version of linux kernel you are using. The line added into ``<profile>.postinstall`` should like below. ::
|
||||
|
||||
/install/postscripts/mlnxofed_ib_install \
|
||||
-p /install/<subpath>/<MLNX_OFED_LINUX.iso> -m --add-kernel-support -end- -i $1 -n genimage
|
||||
|
||||
|
||||
Take rhels7.2 on ppc64le for example: ::
|
||||
|
||||
#lsdef -t osimage rhels7.2-ppc64le-netboot-compute
|
||||
Object name: rhels7.2-ppc64le-netboot-compute
|
||||
exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.exlist
|
||||
imagetype=linux
|
||||
osarch=ppc64le
|
||||
osdistroname=rhels7.2-ppc64le
|
||||
osname=Linux
|
||||
osvers=rhels7.2
|
||||
otherpkgdir=/install/post/otherpkgs/rhels7.2/ppc64le
|
||||
permission=755
|
||||
pkgdir=/install/rhels7.2/ppc64le
|
||||
pkglist=/install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist
|
||||
postinstall=/install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall
|
||||
profile=compute
|
||||
provmethod=netboot
|
||||
rootimgdir=/install/netboot/rhels7.2/ppc64le/compute
|
||||
|
||||
|
||||
**[Note]**: If the osimage definition was generated by xCAT command ``copycds``, default value ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist`` was assigned to ``pkglist`` attribute. ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist`` is the sample pkglist shipped by xCAT, recommend to make a copy of this sample and using the copy in real environment. In the above example, ``/install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist``. For the same reason, ``/install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall``.
|
||||
|
||||
``compute.rhels7.ppc64le.pkglist`` looks like below: ::
|
||||
|
||||
# cat /install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist
|
||||
bash
|
||||
nfs-utils
|
||||
openssl
|
||||
dhclient
|
||||
bc
|
||||
......
|
||||
lsvpd
|
||||
irqbalance
|
||||
procps-ng
|
||||
parted
|
||||
net-tools
|
||||
#INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist#
|
||||
|
||||
``compute.rhels7.ppc64le.ib.postinstall`` looks like below: ::
|
||||
|
||||
# cat /install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall
|
||||
#!/bin/sh
|
||||
#-- Do not remove following line if you want to make use of CVS version tracking
|
||||
.....
|
||||
# [ -r $workdir/$profile.$ext ] && cat $workdir/$profile.$ext | grep -E '^[[:space:]]*#.*[[:space:]]\$Id' >> $installroot/etc/IMGVERSION
|
||||
#done
|
||||
/install/postscripts/mlnxofed_ib_install -p /install/ofed/MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso -i $1 -n genimage
|
||||
|
||||
3. Generate and package image for diskless installation ::
|
||||
|
||||
genimage <osver>-<arch>-netboot-compute
|
||||
packimage <osver>-<arch>-netboot-compute
|
||||
|
||||
4. Install node ::
|
||||
|
||||
nodeset <nodename> osimage=<osver>-<arch>-netboot-compute
|
||||
rsetboot <nodename> net
|
||||
rpower <nodename> reset
|
||||
|
||||
After installation, you can login target ndoe and issue ``ibv_devinfo`` command to verify if your InfiniBand driver works well. if everything is fine, you can get the InfiniBand apater information. ::
|
||||
|
||||
# ibv_devinfo
|
||||
hca_id: mlx5_0
|
||||
transport: InfiniBand (0)
|
||||
fw_ver: 10.14.2036
|
||||
node_guid: f452:1403:0076:10e0
|
||||
sys_image_guid: f452:1403:0076:10e0
|
||||
vendor_id: 0x02c9
|
||||
vendor_part_id: 4113
|
||||
hw_ver: 0x0
|
||||
board_id: IBM1210111019
|
||||
phys_port_cnt: 2
|
||||
Device ports:
|
||||
port: 1
|
||||
state: PORT_INIT (2)
|
||||
max_mtu: 4096 (5)
|
||||
active_mtu: 4096 (5)
|
||||
sm_lid: 0
|
||||
port_lid: 65535
|
||||
port_lmc: 0x00
|
||||
link_layer: InfiniBand
|
||||
|
||||
port: 2
|
||||
state: PORT_DOWN (1)
|
||||
max_mtu: 4096 (5)
|
||||
active_mtu: 4096 (5)
|
||||
sm_lid: 0
|
||||
port_lid: 65535
|
||||
port_lmc: 0x00
|
||||
link_layer: InfiniBand
|
||||
Diskless Installation
|
||||
=====================
|
||||
|
||||
#. Prepare dependency packages in the pkglist
|
||||
|
||||
In order for the Mellanox installation script to execute successfully, certain dependency packages are required to be installed on the compute node. xCAT provides sample package list files to help resolve these dependencies. The samples are located at ``/opt/xcat/share/xcat/ib/netboot/<os>/``.
|
||||
|
||||
To use the ``/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist``, edit your existing ``pkglist`` file for the target osimage and add the following at the bottom: ::
|
||||
|
||||
#INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist#
|
||||
|
||||
#. Configure the ``mlnxofed_ib_install`` script to install the MNLX_OFED drivers
|
||||
|
||||
Edit the ``postinstall`` script on the osimage to invoke the ``mlnxofed_ib_install`` install script.
|
||||
|
||||
For example, take ``rhels7.2-ppc64le-netboot-compute``:
|
||||
|
||||
#. Find the path to the ``postinstall`` script: ::
|
||||
|
||||
# lsdef -t osimage -o rhels7.2-ppc64le-netboot-compute -i postinstall
|
||||
Object name: rhels7.2-ppc64le-netboot-compute
|
||||
postinstall=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall
|
||||
|
||||
#. Edit the ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall`` and add the following: ::
|
||||
|
||||
/install/postscripts/mlnxofed_ib_install \
|
||||
-p /install/<path-to>/<MLNX_OFED_LINUX.iso> -i $1 -n genimage
|
||||
|
||||
*Note: The $1 is a argument that is passed to the the postinstall script at runtime.*
|
||||
|
||||
#. Generate the diskless image
|
||||
|
||||
Use the ``genimage`` command to generate the diskless image from the osimage definition ::
|
||||
|
||||
genimage <osimage>
|
||||
|
||||
Use the ``packimage`` command to pack the diskless image for deployment ::
|
||||
|
||||
packimage <osimage>
|
||||
|
||||
#. Provision the node
|
||||
|
||||
#. Verification
|
||||
|
||||
* The Mellanox IB drivers are located at: ``/lib/modules/<kernel_version>/extra/``
|
||||
|
||||
* Use the ``ibv_devinfo`` comamnd to obtain information about the InfiniBand adapter
|
||||
|
||||
* Check the status of ``openibd`` service
|
||||
|
||||
sysVinit: ::
|
||||
|
||||
service openibd status
|
||||
|
||||
systemd: ::
|
||||
|
||||
systemctl status openibd.service
|
||||
|
||||
|
||||
@ -1,46 +1,59 @@
|
||||
Preparation
|
||||
===========
|
||||
|
||||
Obtain the Mellanox OFED ISO
|
||||
----------------------------
|
||||
|
||||
Obtain the Mellanox OFED ISO file from `Mellanox official Download Page <http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers>`_ and put it into one place under ``/install`` directory depending on your need.
|
||||
|
||||
**[NOTE]**
|
||||
|
||||
* Mellanox provides OFED drivers in **tarball** and **ISO image** formats. xCAT only supports the **iso** format at this time.
|
||||
* Mellanox provides different OFED ISOs depending on operating system and machine architecture, named like MLNX_OFED_LINUX-<packver1>-<packver2>-<osver>-<arch>.iso, you should download correct one according to your environment.
|
||||
* Mellanox has some updates and known issues for echo OFED, please read `InfiniBand/VPI Software Overview <http://www.mellanox.com/page/software_overview_ib>`_ to understand these information.
|
||||
* The Mellanox links offered above maybe outdate in future for Mellanox updates its web page, xCAT will keep updating for synchronization. If we don't update in time, please access `Mellanox web portal <http://www.mellanox.com>`_ to find ``Support/Education`` then ``InfiniBand/VPI Drivers`` lables.
|
||||
|
||||
Prepare Install Script
|
||||
----------------------
|
||||
|
||||
**mlnxofed_ib_install.v2** is a sample script, its framework can help you install Mellanox drives easily. But in specific scenario, some detail need to be modified to meet requirement, such like dependency package list. It has been verified in limited scenarios and can work as solution in these scenarios. For these scenarions information please refer to :doc:`The Scenarios Have Been Verified </advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix>`.
|
||||
|
||||
Copy **mlnxofed_ib_install.v2** into ``/install/postscripts`` and change name to **mlnxofed_ib_install** ::
|
||||
|
||||
cp /opt/xcat/share/xcat/ib/scripts/Mellanox/mlnxofed_ib_install.v2 \
|
||||
/install/postscripts/mlnxofed_ib_install
|
||||
|
||||
chmod +x /install/postscripts/mlnxofed_ib_install
|
||||
|
||||
``mlnxofed_ib_install`` has some options, **'-p' is always needed**.
|
||||
Below are the details of these options:
|
||||
|
||||
* **-p**: [required]--the directory where the OFED iso file is located
|
||||
* **-m**: [optional]--the mlnxofed_ib_install invokes a script ``mlnxofedinstall`` shipped by Mellanox OFED iso. Use this option to pass arguments to the ``mlnxofedinstall``. You must include ``-end-`` at the completion of the options to distinguish the option list. if you don't pass any argument to ``mlnxofedinstall``, **defualt value** ``--without-32bit --without-fw-update --force`` will be passed to ``mlnxofedinstall`` by xCAT.
|
||||
* **-i**: [required for diskless]--the image root path
|
||||
* **-n**: [required for diskless]--nodeset status, the value is 'genimage'
|
||||
|
||||
In general you can use ``mlnxofed_ib_install`` like below ::
|
||||
|
||||
mlnxofed_ib_install -p /install/<path>/<MLNX_OFED_LINUX.iso>
|
||||
|
||||
If need to pass ``--without-32bit --without-fw-update --add-kernel-support --force`` to ``mlnxofedinstall``, refer to below command. ::
|
||||
|
||||
mlnxofed_ib_install -p /install/<path>/<MLNX_OFED_LINUX.iso> \
|
||||
-m --without-32bit --without-fw-update --add-kernel-support --force -end-
|
||||
|
||||
**[Note]** We recommend to update your firmware to the version Mellanox supported in its release notes to avoid unexpected problem when you install InfiniBand driver.
|
||||
|
||||
Preparation
|
||||
===========
|
||||
|
||||
Download MLNX_OFED ISO
|
||||
----------------------
|
||||
|
||||
**xCAT only supports installation using the ISO format.**
|
||||
|
||||
Download the Mellanox OFED ISO file `here (MLNX_OFED) <http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers>`_.
|
||||
|
||||
|
||||
Prepare Installation Script
|
||||
---------------------------
|
||||
|
||||
The ``mlnxofed_ib_install.v2`` is a sample script intended to assist with the installation of the Mellanox OFED drivers. The following support matrix documents the limited number of scenarios that have been verified: :doc:`support matrix </advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix>`.
|
||||
|
||||
#. Copy the ``mlnxofed_ib_install.v2`` to ``/install/postscripts``, renaming to ``mlnxofed_ib_install``. ::
|
||||
|
||||
cp /opt/xcat/share/xcat/ib/scripts/Mellanox/mlnxofed_ib_install.v2 \
|
||||
/install/postscripts/mlnxofed_ib_install
|
||||
|
||||
# ensure the script has execute permission
|
||||
chmod +x /install/postscripts/mlnxofed_ib_install
|
||||
|
||||
#. Familarize the options available for the xCAT ``mlnxofed_ib_install`` script.
|
||||
|
||||
+---------+------------------+----------------------------------------------------------+
|
||||
| Option | Required | Description |
|
||||
+=========+==================+==========================================================+
|
||||
|``-p`` | Yes || The full path to the MLNX_OFED ISO image |
|
||||
+---------+------------------+----------------------------------------------------------+
|
||||
|``-m`` | No || Use this option to pass arguments to the Mellanox OFED |
|
||||
| | || installation script ``mlnxofedinstall``. |
|
||||
| | || |
|
||||
| | || The special keyword ``-end-`` must be added to the end |
|
||||
| | || of the string to mark the completion of the option list |
|
||||
| | || option list. |
|
||||
| | || |
|
||||
| | || If nothing is specified, xCAT passes the the following |
|
||||
| | || ``--without-32bit --with out-fw-update --force`` |
|
||||
+---------+------------------+----------------------------------------------------------+
|
||||
|``-i`` | For diskless || The image root path of the diskless image |
|
||||
| | || |
|
||||
+---------+------------------+----------------------------------------------------------+
|
||||
|``-n`` | For diskless || nodeset status, value is ``genimage`` |
|
||||
| | || |
|
||||
+---------+------------------+----------------------------------------------------------+
|
||||
|
||||
|
||||
A very basic usage of the install script: ::
|
||||
|
||||
/install/postscripts/mlnxofed_ib_install -p /install/<path-to>/<MLNX_OFED_LINUX.iso>
|
||||
|
||||
|
||||
To pass the ``--add-kernel-support`` option to ``mlnxofedinstall``, use the following command: ::
|
||||
|
||||
/install/postscripts/mlnxofed_ib_install -p /install/<path-to>/<MLNX_OFED_LINUX.iso> \
|
||||
-m --without-32bit --without-fw-update --add-kernel-support --force -end-
|
||||
|
||||
|
||||
@ -1,15 +1,52 @@
|
||||
The Scenarios Have Been Verified
|
||||
=================================
|
||||
MLNX_OFED Support Matrix
|
||||
========================
|
||||
|
||||
The following ISO images and attributs have been verified by the xCAT Team.
|
||||
|
||||
**RedHat Enterprise Linux**
|
||||
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| RHEL 7.2 (ppc64le) |
|
||||
+==================================+===================================================================+
|
||||
| **OFED ISO** | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| **Attribute Supported** | --without-32bit --without-fw-update --add-kernel-support --force |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| **IB.pkglist** | ib.rhels7.ppc64le.pkglist |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| RHEL 7.1 (ppc64) |
|
||||
+==================================+===================================================================+
|
||||
| **OFED ISO** | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.1-ppc64.iso |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| **Attribute Supported** | --without-32bit --without-fw-update --add-kernel-support --force |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| **IB.pkglist** | ib.rhels7.ppc64.pkglist |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
|
||||
|
||||
+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+
|
||||
| OS version | Arch | Ofed version | Attribute supported by mlnx | IB.pkglist |
|
||||
+===============+=========+===================================================+==================================================================+===========================+
|
||||
| rhels7.1 | ppc64 | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.1-ppc64.iso |--without-32bit --without-fw-update --add-kernel-support --force | ib.rhels7.ppc64.pkglist |
|
||||
+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+
|
||||
| rhels7.2 | ppc64le | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso |--without-32bit --without-fw-update --add-kernel-support --force | ib.rhels7.ppc64le.pkglist |
|
||||
+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+
|
||||
| sles12.1 | ppc64le |MLNX_OFED_LINUX-3.2-2.0.0.0-sles12sp1-ppc64le.iso |--without-32bit --without-fw-update --force | ib.sles12.ppc64le.pkglist |
|
||||
+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+
|
||||
| ubuntu14.04.3 | ppc64le |MLNX_OFED_LINUX-3.2-2.0.0.0-ubuntu14.04-ppc64le.iso|--without-32bit --without-fw-update --add-kernel-support --force |ib.ubuntu14.ppc64le.pkglist|
|
||||
+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+
|
||||
**Suse Linux Enterprise Server**
|
||||
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| SLES 12 (ppc64le) |
|
||||
+==================================+===================================================================+
|
||||
| **OFED ISO** | MLNX_OFED_LINUX-3.2-2.0.0.0-sles12sp1-ppc64le.iso |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| **Attribute Supported** | --without-32bit --without-fw-update --force |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| **IB.pkglist** | ib.sles12.ppc64le.pkglist |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
|
||||
|
||||
**Ubuntu**
|
||||
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| Ubuntu14.04.3 (ppc64le) |
|
||||
+==================================+===================================================================+
|
||||
| **OFED ISO** | MLNX_OFED_LINUX-3.2-2.0.0.0-ubuntu14.04-ppc64le.iso |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| **Attribute Supported** | --without-32bit --without-fw-update --add-kernel-support --force |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
| **IB.pkglist** | ib.ubuntu14.ppc64le.pkglist |
|
||||
+----------------------------------+-------------------------------------------------------------------+
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user