From ffba03e3bcb195109a32b9710a8c9089b0a330dd Mon Sep 17 00:00:00 2001 From: Victor Hu Date: Wed, 5 Oct 2016 16:04:52 -0400 Subject: [PATCH 1/4] Remove the documentation for mellanox ofed installer version 1. We have version control in the documentation, so there's no need to keep the deprecated documentation in the source --- .../infiniband/driver_and_installation.rst | 11 --- .../advanced/networks/infiniband/index.rst | 9 ++- .../mlnxofed_ib_install_v1_diskful.rst | 53 ------------- .../mlnxofed_ib_install_v1_diskless.rst | 77 ------------------- .../mlnxofed_ib_install_v1_preparation.rst | 42 ---------- .../mlnxofed_ib_install_v1_usage.rst | 9 --- .../infiniband/mlnxofed_ib_install_v2.rst | 14 ++++ .../mlnxofed_ib_install_v2_usage.rst | 12 --- 8 files changed, 21 insertions(+), 206 deletions(-) delete mode 100644 docs/source/advanced/networks/infiniband/driver_and_installation.rst delete mode 100644 docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_diskful.rst delete mode 100644 docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_diskless.rst delete mode 100644 docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_preparation.rst delete mode 100644 docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_usage.rst create mode 100644 docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2.rst delete mode 100644 docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_usage.rst diff --git a/docs/source/advanced/networks/infiniband/driver_and_installation.rst b/docs/source/advanced/networks/infiniband/driver_and_installation.rst deleted file mode 100644 index d9d10a596..000000000 --- a/docs/source/advanced/networks/infiniband/driver_and_installation.rst +++ /dev/null @@ -1,11 +0,0 @@ -IB Driver Preparation and Installation -====================================== - -xCAT provides sample postscripts to help you install the Mellanox OpenFabrics Enterprise Distribution (OFED) InfiniBand Driver. These scripts are located in ``opt/xcat/share/xcat/ib/scripts/Mellanox/``. You can use these scripts directly or change them to satisfy your own environment. **xCAT 2.11 drops support of mlnxofed_ib_install and recommends using version 2 of the script: mlnxofed_ib_install.v2**. - -.. toctree:: - :maxdepth: 2 - - mlnxofed_ib_install_v2_usage.rst - mlnxofed_ib_install_v1_usage.rst - diff --git a/docs/source/advanced/networks/infiniband/index.rst b/docs/source/advanced/networks/infiniband/index.rst index c525d6c95..cb92a7045 100644 --- a/docs/source/advanced/networks/infiniband/index.rst +++ b/docs/source/advanced/networks/infiniband/index.rst @@ -1,13 +1,18 @@ InfiniBand (Mellanox) ===================== -xCAT offers a certain degree support for Mellanox InfiniBand product, it help you to configurate Mellanox InfiniBand products easily. For more information about Mellanox InfiniBand, please refer to `Mellanox official site `_. +xCAT has the ability to help with Mellanox InfiniBand (IB) adapter installation and network configuration as part of the node provisioning process. + .. toctree:: :maxdepth: 2 - driver_and_installation.rst + mlnxofed_ib_install_v2.rst network_configuration.rst switch_configuration.rst ufm_configuration.rst firmware_updates.rst + + + +For more information about Mellanox products, refer to http://www.mellanox.com. diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_diskful.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_diskful.rst deleted file mode 100644 index 37da6def7..000000000 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_diskful.rst +++ /dev/null @@ -1,53 +0,0 @@ -Configuration for Diskful Installation -====================================== - -1. Set script ``mlnxofed_ib_install`` as postbootscript :: - - chdef -p postbootscripts=mlnxofed_ib_install - -2. Specify dependence package **[required for RHEL and SLES]** - - a) Copy the pkglist to the custom directory :: - - cp /opt/xcat/share/xcat/install//compute...pkglist \ - /install/custom/install//compute...pkglist - - b) Edit your /install/custom/install//compute...pkglist and add one line:: - - #INCLUDE:/opt/xcat/share/xcat/ib/netboot//ib...pkglist# - - c) Make the related osimage use the customized pkglist :: - - chdef -t osimage -o --install-compute \ - pkglist=/install/custom/install//compute...pkglist - - Take RHEL 6.4 on x86_64 for example :: - - cp /opt/xcat/share/xcat/install/rh/compute.rhels6.x86_64.pkglist \ - /install/custom/install/rh/compute.rhels6.x86_64.pkglist - - Edit the ``/install/custom/install/rh/compute.rhels6.x86_64.pkglist`` and add below line - ``#INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels6.x86_64.pkglist#`` - - Then ``/install/custom/install/rh/compute.rhels6.x86_64.pkglist`` looks like below :: - - #Please make sure there is a space between @ and group name - #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels6.x86_64.pkglist# - ntp - nfs-utils - net-snmp - rsync - yp-tools - openssh-server - util-linux-ng - - Then modify related osimage :: - - chdef -t osimage -o rhels6.4-x86_64-install-compute \ - pkglist=/install/custom/install/rh/compute.rhels6.x86_64.pkglist - -3. Install node :: - - nodeset osimage=--install-compute - rsetboot net - rpower reset diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_diskless.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_diskless.rst deleted file mode 100644 index fd1173186..000000000 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_diskless.rst +++ /dev/null @@ -1,77 +0,0 @@ -Configuration for Diskless Installation -======================================= - -1. Specify dependence package **[required for RHEL and SLES]** - - a) Copy a correct pkglist file **shipped by xCAT** according your environment to the ``/install/custom/netboot//`` directory :: - - cp /opt/xcat/share/xcat/netboot//compute...pkglist \ - /install/custom/netboot//compute...pkglist - - b) Edit your ``/install/custom/netboot//.pkglist`` and add one line ``#INCLUDE:/opt/xcat/share/xcat/ib/netboot//ib...pkglist#`` - - Take RHEL 6.4 on x86_64 for example :: - - cp /opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.pkglist \ - /install/custom/netboot/rh/compute.rhels6.x86_64.pkglist - - Edit the ``/install/custom/netboot/rh/compute.rhels6.x86_64.pkglist`` and add below line - ``#INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels6.x86_64.pkglist#`` - - Then ``/install/custom/netboot/rh/compute.rhels6.x86_64.pkglist`` looks like below :: - - #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels6.x86_64.pkglist# - bash - nfs-utils - openssl - dhclient - ..... - - -2. Prepare postinstall scripts - - a) Specify postinstall script **shipped by xCAT** :: - - mkdir -p /install/custom/netboot// - - cp /opt/xcat/share/xcat/netboot//.postinstall \ - /install/custom/netboot// - - chmod +x /install/custom/netboot//.postinstall - - Take RHEL 6.4 on x86_64 for example :: - - mkdir -p /install/custom/netboot/rh/ - cp /opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.postinstall \ - /install/custom/netboot/rh/ - chmod +x /install/custom/netboot/rh/compute.rhels6.x86_64.postinstall - - b) Edit ``/install/custom/netboot//.postinstall`` and add below line in the end: :: - - installroot=$1 ofeddir=/install/post/otherpkgs///ofed/ \ - NODESETSTATE=genimage mlnxofed_options=--force /install/postscripts/mlnxofed_ib_install - - -3. Set the related osimage use the customized pkglist and customized compute.postinsall - -* [RHEL/SLES] :: - - chdef -t osimage -o --netboot-compute \ - pkglist=/install/custom/netboot//compute...pkglist \ - postinstall=/install/custom/netboot//.postinstall - -* [Ubuntu] :: - - chdef -t osimage -o --netboot-compute \ - postinstall=/install/custom/netboot//.postinstall - -4. Generate and package image for diskless installation :: - - genimage --netboot-compute - packimage --netboot-compute - -5. Install node :: - - nodeset osimage=--netboot-compute - rsetboot net - rpower reset diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_preparation.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_preparation.rst deleted file mode 100644 index 340871bef..000000000 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_preparation.rst +++ /dev/null @@ -1,42 +0,0 @@ -Preparation -=========== - -Obtain the Mellanox OFED ISO file from `Mellanox official site `_ and mount it onto suggested target location on the xCAT MN according your OS and ARCH: :: - - mkdir -p /install/post/otherpkgs///ofed - - mount -o loop MLNX_OFED_LINUX----.iso \ - /install/post/otherpkgs///ofed - -Take sles11 sp1 for x86_64 as an example :: - - mkdir -p /install/post/otherpkgs/sles11.1/x86_64/ofed/ - - mount -o loop MLNX_OFED_LINUX-1.5.3-3.0.0-sles11sp1-x86_64.iso \ - /install/post/otherpkgs/sles11.1/x86_64/ofed/ - - -Take Ubuntu14.4.1 for Power8 LE as an example :: - - mkdir -p /install/post/otherpkgs/ubuntu14.04.1/ppc64el/ofed - - mount -o loop MLNX_OFED_LINUX-2.3-1.0.1-ubuntu14.04-ppc64le.iso \ - /install/post/otherpkgs/ubuntu14.04.1/ppc64el/ofed - - -**[NOTE]** - -* Mellanox provides OFED files with **tarball** and **ISO** two format, but for xCAT, we just support **ISO** format right now. - -Copy Sample script **mlnxofed_ib_install** shipped by xCAT into ``/install/postscripts`` before using, such as :: - - cp /opt/xcat/share/xcat/ib/scripts/Mellanox/mlnxofed_ib_install \ - /install/postscripts/mlnxofed_ib_install - -The **mlnxofed_ib_install** invokes a script ``mlnxofedinstall`` shipped by Mellanox OFED ISO. If you want to pass the argument to ``mlnxofedinstall``, you set the argument to the environment variable ``mlnxofed_options`` which could be read by **mlnxofed_ib_install**. For example: PPE requires the 32-bit version of libibverbs, but the default **mlnxofed_ib_install** will remove all the old ib related packages at first including the 32-bit version of libibverbs. In this case, you can set the environment variable ``mlnxofed_options=--force`` when running the **mlnxofed_ib_install**. For diskful, you should put the environment variable ``mlnxofed_options=--force`` in mypostscript.tmpl. myposcript.tmpl is in ``/opt/xcat/share/xcat/templates/mypostscript/`` by default. When customize it, you should copy it into ``/install/postscripts/myposcript.tmpl`` :: - - mlnxofed_options='--force' - export mlnxofed_options - - - \ No newline at end of file diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_usage.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_usage.rst deleted file mode 100644 index 79577bdbd..000000000 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v1_usage.rst +++ /dev/null @@ -1,9 +0,0 @@ -Using mlnxofed_ib_install (drops support) -========================================= - -.. toctree:: - :maxdepth: 1 - - mlnxofed_ib_install_v1_preparation.rst - mlnxofed_ib_install_v1_diskful.rst - mlnxofed_ib_install_v1_diskless.rst \ No newline at end of file diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2.rst new file mode 100644 index 000000000..493ab2b95 --- /dev/null +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2.rst @@ -0,0 +1,14 @@ +Mellanox OFED Installation Script +================================= + +Mellanox provides a tested and packaged version of the OpenFabrics Enterprise Distribution (OFED) driver, named Mellanox OFED (MLNX_OFED). To assist with the installation of the MLNX_OFED driver, xCAT provids a sample postscript: ``mlnxofed_ib_install.v2``. + +.. toctree:: + :maxdepth: 2 + + mlnxofed_ib_install_v2_preparation.rst + mlnxofed_ib_install_v2_diskful.rst + mlnxofed_ib_install_v2_diskless.rst + mlnxofed_ib_verified_scenario_matrix.rst + mlnxofed_ib_known_issue.rst + diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_usage.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_usage.rst deleted file mode 100644 index 52d500fb3..000000000 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_usage.rst +++ /dev/null @@ -1,12 +0,0 @@ -Using mlnxofed_ib_install.v2 (Recommend) -========================================= - -.. toctree:: - :maxdepth: 1 - - mlnxofed_ib_install_v2_preparation.rst - mlnxofed_ib_install_v2_diskful.rst - mlnxofed_ib_install_v2_diskless.rst - mlnxofed_ib_verified_scenario_matrix.rst - mlnxofed_ib_known_issue.rst - From e10b5afad320dfa661d1c6ce86deb47c6f294d97 Mon Sep 17 00:00:00 2001 From: Victor Hu Date: Wed, 5 Oct 2016 16:05:50 -0400 Subject: [PATCH 2/4] Refactor the mellanox installation instructions to make it easier to follow --- .../infiniband/mlnxofed_configuration.rst | 11 + .../infiniband/mlnxofed_ib_install_v2.rst | 3 +- .../mlnxofed_ib_install_v2_diskful.rst | 207 +++------------- .../mlnxofed_ib_install_v2_diskless.rst | 227 +++++------------- .../mlnxofed_ib_install_v2_preparation.rst | 105 ++++---- .../mlnxofed_ib_verified_scenario_matrix.rst | 63 ++++- 6 files changed, 217 insertions(+), 399 deletions(-) create mode 100644 docs/source/advanced/networks/infiniband/mlnxofed_configuration.rst diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_configuration.rst b/docs/source/advanced/networks/infiniband/mlnxofed_configuration.rst new file mode 100644 index 000000000..24e9e912a --- /dev/null +++ b/docs/source/advanced/networks/infiniband/mlnxofed_configuration.rst @@ -0,0 +1,11 @@ +Configuration +============= + +The process to configure the osimage to install the Mellanox OFED Drivers for Diskful and Diskless scenarios are outlined below. + +.. toctree:: + :maxdepth: 2 + + mlnxofed_ib_install_v2_diskful.rst + mlnxofed_ib_install_v2_diskless.rst + diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2.rst index 493ab2b95..5658f2d1b 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2.rst @@ -7,8 +7,7 @@ Mellanox provides a tested and packaged version of the OpenFabrics Enterprise Di :maxdepth: 2 mlnxofed_ib_install_v2_preparation.rst - mlnxofed_ib_install_v2_diskful.rst - mlnxofed_ib_install_v2_diskless.rst + mlnxofed_configuration.rst mlnxofed_ib_verified_scenario_matrix.rst mlnxofed_ib_known_issue.rst diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst index 453e888a5..ac25a5566 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst @@ -1,169 +1,38 @@ -Configuration for Diskful Installation -======================================= - -1. Set script ``mlnxofed_ib_install`` as ``postbootscripts`` or ``postscripts`` :: - - chdef -p postbootscripts="mlnxofed_ib_install -p /install//" - - Or :: - - chdef -p postscripts="mlnxofed_ib_install -p /install//" - - xCAT simulates completely the way Mellanox scripts work by using ``postbootscripts``. This way need to reboot after drive installation to make Mellanox drivers work reliably just like Mellanox suggested. If you want to use the reboot after operating system installation to avoid reboot twice, you can using ``postscripts`` attribute to install Mellanox drivers. This way has been verified in limited scenarios. For more information please refer to :doc:`The Scenarios Have Been Verified `. You can try this way in other else scenarios if you needed. - -2. Specify dependency package - - Some dependencies need to be installed before running Mellanox scripts. These dependencies are different between different scenario. xCAT configurates these dependency packages by using ``pkglist`` attribute of ``osimage`` definition. Please refer to :doc:`Add Additional Software Packages ` for more information:: - - # lsdef -t osimage --install-compute - Object name: --install-compute - imagetype=linux - .... - pkgdir=/ - pkglist=//compute...pkglist - .... - - You can append the ib dependency packages list in the end of ``//compute...pkglist`` directly like below: :: - - #cat //compute...pkglist - @base - @x11 - openssl - ntp - rsyn - #ib part - createrepo - kernel-devel - kernel-source - .... - - - Or if you want to isolate InfiniBand dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute...pkglist`` like below way: :: - - #cat //compute...pkglist - @base - @x11 - openssl - ntp - rsyn - #INCLUDE://# - - xCAT has shipped some ib pkglist files under ``/opt/xcat/share/xcat/ib/netboot//``, these pkglist files have been verified in sepecific scenario. Please refer to :doc:`The Scenarios Have Been Verified ` to judge if you can use it directly in your environment. If so, you can use it like below: :: - - #cat //compute...pkglist - @base - @x11 - openssl - ntp - rsyn - #INCLUDE:/opt/xcat/share/xcat/ib/netboot//ib...pkglist# - - Take rhels7.2 on ppc64le for example: :: - - # lsdef -t osimage rhels7.2-ppc64le-install-compute - Object name: rhels7.2-ppc64le-install-compute - imagetype=linux - osarch=ppc64le - osdistroname=rhels7.2-ppc64le - osname=Linux - osvers=rhels7.2 - otherpkgdir=/install/post/otherpkgs/rhels7.2/ppc64le - pkgdir=/install/rhels7.2/ppc64le - pkglist=/install/custom/install/rh/compute.rhels7.ib.pkglist - profile=compute - provmethod=install - template=/opt/xcat/share/xcat/install/rh/compute.rhels7.tmpl - - - **[Note]**: If the osimage definition was generated by xCAT command ``copycds``, default value ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` was assigned to ``pkglist`` attribute. ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` is the sample pkglist shipped by xCAT, recommend to make a copy of this sample and using the copy in real environment. In the above example, ``/install/custom/install/rh/compute.rhels7.ib.pkglist`` is a copy of ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist``. :: - - # cat /install/custom/install/rh/compute.rhels7.ib.pkglist - #Please make sure there is a space between @ and group name - wget - ntp - nfs-utils - net-snmp - rsync - yp-tools - openssh-server - util-linux - net-tools - #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist# - - - -3. Install node :: - - nodeset osimage=--install-compute - rsetboot net - rpower reset - - - After steps above, you can login target node and find the Mellanox InfiniBand drives are located under ``/lib/modules//extra/``. - - Issue ``ibv_devinfo`` command you can get the InfiniBand apater information :: - - # ibv_devinfo - hca_id: mlx5_0 - transport: InfiniBand (0) - fw_ver: 10.14.2036 - node_guid: f452:1403:0076:10e0 - sys_image_guid: f452:1403:0076:10e0 - vendor_id: 0x02c9 - vendor_part_id: 4113 - hw_ver: 0x0 - board_id: IBM1210111019 - phys_port_cnt: 2 - Device ports: - port: 1 - state: PORT_INIT (2) - max_mtu: 4096 (5) - active_mtu: 4096 (5) - sm_lid: 0 - port_lid: 65535 - port_lmc: 0x00 - link_layer: InfiniBand - - port: 2 - state: PORT_DOWN (1) - max_mtu: 4096 (5) - active_mtu: 4096 (5) - sm_lid: 0 - port_lid: 65535 - port_lmc: 0x00 - link_layer: InfiniBand - - Using ``service openibd status`` to verify if openibd works well. Below is the output in rhels7.2. :: - - - # service openibd status - HCA driver loaded - - Configured IPoIB devices: - ib0 ib1 - - Currently active IPoIB devices: - Configured Mellanox EN devices: - - Currently active Mellanox devices: - - The following OFED modules are loaded: - - rdma_ucm - rdma_cm - ib_addr - ib_ipoib - mlx4_core - mlx4_ib - mlx4_en - mlx5_core - mlx5_ib - ib_uverbs - ib_umad - ib_ucm - ib_sa - ib_cm - ib_mad - ib_core - - +Diskful Installation +==================== + +#. Prepare dependency packages in the pkglist + + In order for the Mellanox installation script to execute successfully, certain dependency packages are required to be installed on the compute node. xCAT provides sample package list files to help resolve these dependencies. The samples are located at ``/opt/xcat/share/xcat/ib/netboot//``. + + To use the ``/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist``, edit your existing ``pkglist`` file for the target osimage and add the following at the bottom: :: + + #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist# + +#. Configure the ``mlnxofed_ib_install`` script to install the MNLX_OFED drivers + + xCAT has a concept of postscripts that can be used to customize the node after the operating system is installed. + + Mellanox recommends that the operating system is rebooted after the drivers are installed, so xCAT recommends using the ``postscripts`` attribute to avoid the need for a second reboot. To invoke the ``mlnxofed_ib_install`` as a postscript :: + + chdef -t node -o \ + -p postscripts="mlnxofed_ib_install -p /install//" + +#. Provision the node + +#. Verification + + * The Mellanox IB drivers are located at: ``/lib/modules//extra/`` + + * Use the ``ibv_devinfo`` comamnd to obtain information about the InfiniBand adapter + + * Check the status of ``openibd`` service + + sysVinit: :: + + service openibd status + + systemd: :: + + systemctl status openibd.service + diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst index e9576024f..97f8face4 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst @@ -1,169 +1,58 @@ -Configuration for Diskless Installation -======================================= - -1. Specify dependency package - - Some dependencies need to be installed before running Mellanox scripts. These dependencies are different among different scenarios. xCAT can help user to install these dependency packages by adding these package names to the file specified by the ``pkglist`` attribute of the ``osimage`` definition. Please refer to :doc:`Add Additional Software Packages ` for more information:: - - # lsdef -t osimage --netboot-compute - Object name: --netboot-compute - imagetype=linux - .... - pkgdir=/ - pkglist=//compute...pkglist - .... - - You can append the ib dependency packages list in the end of ``//compute...pkglist`` directly like below: :: - - #cat //compute...pkglist - bash - nfs-utils - openssl - dhclient - kernel - ..... - - #ib part - createrepo - kernel-devel - kernel-source - .... - - - Or if you want to isolate InfiniBand dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute...pkglist`` like below way: :: - - #cat //compute...pkglist - bash - nfs-utils - openssl - dhclient - kernel - ..... - #INCLUDE://# - - xCAT ships some InfiniBand pkglist files under ``/opt/xcat/share/xcat/ib/netboot//``, these pkglist files have been verified in sepecific scenario. Please refer to :doc:`The Scenarios Have Been Verified ` to judge if you can use it directly in your environment. If so, you can use it like below: :: - - #cat //compute...pkglist - bash - nfs-utils - openssl - dhclient - kernel - ..... - #INCLUDE:/opt/xcat/share/xcat/ib/netboot//ib...pkglist# - - -2. Prepare postinstall scripts - - Edit ``postinstall`` script to trigger InfniBand drvices installation during ``genimage``. Using below command to find out where the ``postinstall`` script is defined. :: - - # lsdef -t osimage --netboot-compute - Object name: --netboot-compute - .... - postinstall=/..postinstall - .... - - - - Edit ``/..postinstall`` and add below line in the end. :: - - /install/postscripts/mlnxofed_ib_install \ - -p /install// -i $1 -n genimage - - - **[Note]** Mellanox OFED ISO was built against a series of certain kernael versions, If the version of linux kernel you are using does not match with any of the Mellanox offered pre-built kernel modules, you can pass ``--add-kernel-support`` command line argument to Mellanox OFED installation script to build these kernel modules base on the version of linux kernel you are using. The line added into ``.postinstall`` should like below. :: - - /install/postscripts/mlnxofed_ib_install \ - -p /install// -m --add-kernel-support -end- -i $1 -n genimage - - - Take rhels7.2 on ppc64le for example: :: - - #lsdef -t osimage rhels7.2-ppc64le-netboot-compute - Object name: rhels7.2-ppc64le-netboot-compute - exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.exlist - imagetype=linux - osarch=ppc64le - osdistroname=rhels7.2-ppc64le - osname=Linux - osvers=rhels7.2 - otherpkgdir=/install/post/otherpkgs/rhels7.2/ppc64le - permission=755 - pkgdir=/install/rhels7.2/ppc64le - pkglist=/install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist - postinstall=/install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall - profile=compute - provmethod=netboot - rootimgdir=/install/netboot/rhels7.2/ppc64le/compute - - - **[Note]**: If the osimage definition was generated by xCAT command ``copycds``, default value ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist`` was assigned to ``pkglist`` attribute. ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist`` is the sample pkglist shipped by xCAT, recommend to make a copy of this sample and using the copy in real environment. In the above example, ``/install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist``. For the same reason, ``/install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall``. - - ``compute.rhels7.ppc64le.pkglist`` looks like below: :: - - # cat /install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist - bash - nfs-utils - openssl - dhclient - bc - ...... - lsvpd - irqbalance - procps-ng - parted - net-tools - #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist# - - ``compute.rhels7.ppc64le.ib.postinstall`` looks like below: :: - - # cat /install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall - #!/bin/sh - #-- Do not remove following line if you want to make use of CVS version tracking - ..... - # [ -r $workdir/$profile.$ext ] && cat $workdir/$profile.$ext | grep -E '^[[:space:]]*#.*[[:space:]]\$Id' >> $installroot/etc/IMGVERSION - #done - /install/postscripts/mlnxofed_ib_install -p /install/ofed/MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso -i $1 -n genimage - -3. Generate and package image for diskless installation :: - - genimage --netboot-compute - packimage --netboot-compute - -4. Install node :: - - nodeset osimage=--netboot-compute - rsetboot net - rpower reset - - After installation, you can login target ndoe and issue ``ibv_devinfo`` command to verify if your InfiniBand driver works well. if everything is fine, you can get the InfiniBand apater information. :: - - # ibv_devinfo - hca_id: mlx5_0 - transport: InfiniBand (0) - fw_ver: 10.14.2036 - node_guid: f452:1403:0076:10e0 - sys_image_guid: f452:1403:0076:10e0 - vendor_id: 0x02c9 - vendor_part_id: 4113 - hw_ver: 0x0 - board_id: IBM1210111019 - phys_port_cnt: 2 - Device ports: - port: 1 - state: PORT_INIT (2) - max_mtu: 4096 (5) - active_mtu: 4096 (5) - sm_lid: 0 - port_lid: 65535 - port_lmc: 0x00 - link_layer: InfiniBand - - port: 2 - state: PORT_DOWN (1) - max_mtu: 4096 (5) - active_mtu: 4096 (5) - sm_lid: 0 - port_lid: 65535 - port_lmc: 0x00 - link_layer: InfiniBand +Diskless Installation +===================== + +#. Prepare dependency packages in the pkglist + + In order for the Mellanox installation script to execute successfully, certain dependency packages are required to be installed on the compute node. xCAT provides sample package list files to help resolve these dependencies. The samples are located at ``/opt/xcat/share/xcat/ib/netboot//``. + + To use the ``/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist``, edit your existing ``pkglist`` file for the target osimage and add the following at the bottom: :: + + #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist# + +#. Configure the ``mlnxofed_ib_install`` script to install the MNLX_OFED drivers + + Edit the ``postinstall`` script on the osimage to invoke the ``mlnxofed_ib_install`` install script. + + For example, take ``rhels7.2-ppc64le-netboot-compute``: + + #. Find the path to the ``postinstall`` script: :: + + # lsdef -t osimage -o rhels7.2-ppc64le-netboot-compute -i postinstall + Object name: rhels7.2-ppc64le-netboot-compute + postinstall=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall + + #. Edit the ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall`` and add the following: :: + + /install/postscripts/mlnxofed_ib_install \ + -p /install// -i $1 -n genimage + + *Note: The $1 is a argument that is passed to the the postinstall script at runtime.* + +#. Generate the diskless image + + Use the ``genimage`` command to generate the diskless image from the osimage definition :: + + genimage + + Use the ``packimage`` command to pack the diskless image for deployment :: + + packimage + +#. Provision the node + +#. Verification + + * The Mellanox IB drivers are located at: ``/lib/modules//extra/`` + + * Use the ``ibv_devinfo`` comamnd to obtain information about the InfiniBand adapter + + * Check the status of ``openibd`` service + + sysVinit: :: + + service openibd status + + systemd: :: + + systemctl status openibd.service + diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst index cff51e6a6..929bdf674 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst @@ -1,46 +1,59 @@ -Preparation -=========== - -Obtain the Mellanox OFED ISO ----------------------------- - -Obtain the Mellanox OFED ISO file from `Mellanox official Download Page `_ and put it into one place under ``/install`` directory depending on your need. - -**[NOTE]** - -* Mellanox provides OFED drivers in **tarball** and **ISO image** formats. xCAT only supports the **iso** format at this time. -* Mellanox provides different OFED ISOs depending on operating system and machine architecture, named like MLNX_OFED_LINUX----.iso, you should download correct one according to your environment. -* Mellanox has some updates and known issues for echo OFED, please read `InfiniBand/VPI Software Overview `_ to understand these information. -* The Mellanox links offered above maybe outdate in future for Mellanox updates its web page, xCAT will keep updating for synchronization. If we don't update in time, please access `Mellanox web portal `_ to find ``Support/Education`` then ``InfiniBand/VPI Drivers`` lables. - -Prepare Install Script ----------------------- - -**mlnxofed_ib_install.v2** is a sample script, its framework can help you install Mellanox drives easily. But in specific scenario, some detail need to be modified to meet requirement, such like dependency package list. It has been verified in limited scenarios and can work as solution in these scenarios. For these scenarions information please refer to :doc:`The Scenarios Have Been Verified `. - -Copy **mlnxofed_ib_install.v2** into ``/install/postscripts`` and change name to **mlnxofed_ib_install** :: - - cp /opt/xcat/share/xcat/ib/scripts/Mellanox/mlnxofed_ib_install.v2 \ - /install/postscripts/mlnxofed_ib_install - - chmod +x /install/postscripts/mlnxofed_ib_install - -``mlnxofed_ib_install`` has some options, **'-p' is always needed**. -Below are the details of these options: - -* **-p**: [required]--the directory where the OFED iso file is located -* **-m**: [optional]--the mlnxofed_ib_install invokes a script ``mlnxofedinstall`` shipped by Mellanox OFED iso. Use this option to pass arguments to the ``mlnxofedinstall``. You must include ``-end-`` at the completion of the options to distinguish the option list. if you don't pass any argument to ``mlnxofedinstall``, **defualt value** ``--without-32bit --without-fw-update --force`` will be passed to ``mlnxofedinstall`` by xCAT. -* **-i**: [required for diskless]--the image root path -* **-n**: [required for diskless]--nodeset status, the value is 'genimage' - -In general you can use ``mlnxofed_ib_install`` like below :: - - mlnxofed_ib_install -p /install// - -If need to pass ``--without-32bit --without-fw-update --add-kernel-support --force`` to ``mlnxofedinstall``, refer to below command. :: - - mlnxofed_ib_install -p /install// \ - -m --without-32bit --without-fw-update --add-kernel-support --force -end- - -**[Note]** We recommend to update your firmware to the version Mellanox supported in its release notes to avoid unexpected problem when you install InfiniBand driver. - +Preparation +=========== + +Download MLNX_OFED ISO +---------------------- + +**xCAT only supports installation using the ISO format.** + +Download the Mellanox OFED ISO file `here (MLNX_OFED) `_. + + +Prepare Installation Script +--------------------------- + +The ``mlnxofed_ib_install.v2`` is a sample script intended to assist with the installation of the Mellanox OFED drivers. The following support matrix documents the limited number of scenarios that have been verified: :doc:`support matrix `. + +#. Copy the ``mlnxofed_ib_install.v2`` to ``/install/postscripts``, renaming to ``mlnxofed_ib_install``. :: + + cp /opt/xcat/share/xcat/ib/scripts/Mellanox/mlnxofed_ib_install.v2 \ + /install/postscripts/mlnxofed_ib_install + + # ensure the script has execute permission + chmod +x /install/postscripts/mlnxofed_ib_install + +#. Familarize the options available for the xCAT ``mlnxofed_ib_install`` script. + + +---------+------------------+----------------------------------------------------------+ + | Option | Required | Description | + +=========+==================+==========================================================+ + |``-p`` | Yes || The full path to the MLNX_OFED ISO image | + +---------+------------------+----------------------------------------------------------+ + |``-m`` | No || Use this option to pass arguments to the Mellanox OFED | + | | || installation script ``mlnxofedinstall``. | + | | || | + | | || The special keyword ``-end-`` must be added to the end | + | | || of the string to mark the completion of the option list | + | | || option list. | + | | || | + | | || If nothing is specified, xCAT passes the the following | + | | || ``--without-32bit --with out-fw-update --force`` | + +---------+------------------+----------------------------------------------------------+ + |``-i`` | For diskless || The image root path of the diskless image | + | | || | + +---------+------------------+----------------------------------------------------------+ + |``-n`` | For diskless || nodeset status, value is ``genimage`` | + | | || | + +---------+------------------+----------------------------------------------------------+ + + + A very basic usage of the install script: :: + + /install/postscripts/mlnxofed_ib_install -p /install// + + + To pass the ``--add-kernel-support`` option to ``mlnxofedinstall``, use the following command: :: + + /install/postscripts/mlnxofed_ib_install -p /install// \ + -m --without-32bit --without-fw-update --add-kernel-support --force -end- + diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix.rst index d8a515de6..fd9bfded6 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix.rst @@ -1,15 +1,52 @@ -The Scenarios Have Been Verified -================================= +MLNX_OFED Support Matrix +======================== + +The following ISO images and attributs have been verified by the xCAT Team. + +**RedHat Enterprise Linux** + ++----------------------------------+-------------------------------------------------------------------+ +| RHEL 7.2 (ppc64le) | ++==================================+===================================================================+ +| **OFED ISO** | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso | ++----------------------------------+-------------------------------------------------------------------+ +| **Attribute Supported** | --without-32bit --without-fw-update --add-kernel-support --force | ++----------------------------------+-------------------------------------------------------------------+ +| **IB.pkglist** | ib.rhels7.ppc64le.pkglist | ++----------------------------------+-------------------------------------------------------------------+ + ++----------------------------------+-------------------------------------------------------------------+ +| RHEL 7.1 (ppc64) | ++==================================+===================================================================+ +| **OFED ISO** | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.1-ppc64.iso | ++----------------------------------+-------------------------------------------------------------------+ +| **Attribute Supported** | --without-32bit --without-fw-update --add-kernel-support --force | ++----------------------------------+-------------------------------------------------------------------+ +| **IB.pkglist** | ib.rhels7.ppc64.pkglist | ++----------------------------------+-------------------------------------------------------------------+ -+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ -| OS version | Arch | Ofed version | Attribute supported by mlnx | IB.pkglist | -+===============+=========+===================================================+==================================================================+===========================+ -| rhels7.1 | ppc64 | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.1-ppc64.iso |--without-32bit --without-fw-update --add-kernel-support --force | ib.rhels7.ppc64.pkglist | -+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ -| rhels7.2 | ppc64le | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso |--without-32bit --without-fw-update --add-kernel-support --force | ib.rhels7.ppc64le.pkglist | -+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ -| sles12.1 | ppc64le |MLNX_OFED_LINUX-3.2-2.0.0.0-sles12sp1-ppc64le.iso |--without-32bit --without-fw-update --force | ib.sles12.ppc64le.pkglist | -+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ -| ubuntu14.04.3 | ppc64le |MLNX_OFED_LINUX-3.2-2.0.0.0-ubuntu14.04-ppc64le.iso|--without-32bit --without-fw-update --add-kernel-support --force |ib.ubuntu14.ppc64le.pkglist| -+---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ +**Suse Linux Enterprise Server** + ++----------------------------------+-------------------------------------------------------------------+ +| SLES 12 (ppc64le) | ++==================================+===================================================================+ +| **OFED ISO** | MLNX_OFED_LINUX-3.2-2.0.0.0-sles12sp1-ppc64le.iso | ++----------------------------------+-------------------------------------------------------------------+ +| **Attribute Supported** | --without-32bit --without-fw-update --force | ++----------------------------------+-------------------------------------------------------------------+ +| **IB.pkglist** | ib.sles12.ppc64le.pkglist | ++----------------------------------+-------------------------------------------------------------------+ + + +**Ubuntu** + ++----------------------------------+-------------------------------------------------------------------+ +| Ubuntu14.04.3 (ppc64le) | ++==================================+===================================================================+ +| **OFED ISO** | MLNX_OFED_LINUX-3.2-2.0.0.0-ubuntu14.04-ppc64le.iso | ++----------------------------------+-------------------------------------------------------------------+ +| **Attribute Supported** | --without-32bit --without-fw-update --add-kernel-support --force | ++----------------------------------+-------------------------------------------------------------------+ +| **IB.pkglist** | ib.ubuntu14.ppc64le.pkglist | ++----------------------------------+-------------------------------------------------------------------+ From 87b8b1ea5288e39acf42ec223f50937224f6b87b Mon Sep 17 00:00:00 2001 From: Victor Hu Date: Fri, 7 Oct 2016 14:05:34 -0400 Subject: [PATCH 3/4] Reformat some of the information in the known issues --- .../infiniband/mlnxofed_ib_known_issue.rst | 29 +++++++++++++------ 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst index c523de4fa..2cd9614d6 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst @@ -1,19 +1,30 @@ Known Issues ============ +Kernel Mismatch +--------------- -Known Issue 1 -------------- +The Mellanox OFED ISO is built againt a series of specific kernel version. If the version of the linux kernel does not match any of the Mellanox offered pre-built kernel modules, you can pass the ``--add-kernel-support`` argument to the Mellanox installation script to build the kernel modules based on the version you are using. + + For diskless images, add the following line to the ``postinstall`` script: :: + + /install/postscripts/mlnxofed_ib_install \ + -p /install// -m --add-kernel-support -end- \ + -i $1 -n genimage + +Preventing upgrade of the Mellanox Drivers +------------------------------------------ + +On RedHat operating systems, after the Mellanox drivers are installed, you may have a requirement to update your operating system to a later version. +Some operating systems may ship InfiniBand drivers that are higher version than the Mellanox drivers you have installed and therefor may update the existing drivers. + +To prevent this from happening, add the following in the ``/etc/yum.conf`` :: -After you install mellanox derives in rhels7.2 successfully by xCAT, maybe you have new requirement to upgrade your operating system to higher version. In this case you probably hit such problem the IB adaptor drives shipped by operating system is higher than the Mellanox drives you have installed. That means the the Mellanox drives will be replaced by the drives shipped by operating system. If it's not the result you expect, you hope keep the Mellanox drives after operating system upgraded, please add below statement into ``/etc/yum.conf`` in your target node after you install mellanox derives successfully for the first time. :: - exclude=dapl* libib* ibacm infiniband* libmlx* librdma* opensm* ibutils* -Known Issue 2 -------------- +Development packages in SLES +---------------------------- -If you want to use ``--add-kernel-support`` attribute in sles12.1 and ppc64le scenario, you will find some dependency packages are not shipped by SLES Server DVDs, such like ``python-devel``, it's shipped in SDK DVDs. xCAT doesn't ship specific pkglist to support such scenario. If you have such requirement, please used ``otherpkglist`` and ``otherpkgs`` attributes to prepare dependency packages repository ahead. If you need help about ``otherpkglist`` and ``otherpkgs`` attributes, please refer to :doc:`Add Additional Software Packages `. +If using the ``--add-kernel-support`` attribute on SLES operating systems, you may find problems with installing some dependency packages which are not shipped by the SLES server DVDs. The development rpms are provided by the SDK DVDs. Refer to :doc:`Add Additional Software Packages ` to configure the SDK repositories. - - From ad4d60627aa3d7733de24e873b3fa8c2dbb88622 Mon Sep 17 00:00:00 2001 From: Victor Hu Date: Fri, 7 Oct 2016 15:56:12 -0400 Subject: [PATCH 4/4] Move the kernel mismatch issue back to the diskless and diskful sections --- .../infiniband/mlnxofed_ib_install_v2_diskful.rst | 6 ++++++ .../infiniband/mlnxofed_ib_install_v2_diskless.rst | 6 ++++++ .../networks/infiniband/mlnxofed_ib_known_issue.rst | 11 ----------- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst index ac25a5566..0db6df69e 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst @@ -18,6 +18,12 @@ Diskful Installation chdef -t node -o \ -p postscripts="mlnxofed_ib_install -p /install//" + **[kernel mismatch issue]** The Mellanox OFED ISO is built againt a series of specific kernel version. If the version of the linux kernel does not match any of the Mellanox offered pre-built kernel modules, you can pass the ``--add-kernel-support`` argument to the Mellanox installation script to build the kernel modules based on the version you are using. :: + + chdef -t node -o \ + -p postscripts="mlnxofed_ib_install -p /install// \ + -m --add-kernel-support -end-" + #. Provision the node #. Verification diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst index 97f8face4..ae1be14ca 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst @@ -27,6 +27,12 @@ Diskless Installation -p /install// -i $1 -n genimage *Note: The $1 is a argument that is passed to the the postinstall script at runtime.* + + **[kernel mismatch issue]** The Mellanox OFED ISO is built againt a series of specific kernel version. If the version of the linux kernel does not match any of the Mellanox offered pre-built kernel modules, you can pass the ``--add-kernel-support`` argument to the Mellanox installation script to build the kernel modules based on the version you are using. :: + + /install/postscripts/mlnxofed_ib_install \ + -p /install// -m --add-kernel-support -end- \ + -i $1 -n genimage #. Generate the diskless image diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst index 2cd9614d6..2d337a588 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst @@ -1,17 +1,6 @@ Known Issues ============ -Kernel Mismatch ---------------- - -The Mellanox OFED ISO is built againt a series of specific kernel version. If the version of the linux kernel does not match any of the Mellanox offered pre-built kernel modules, you can pass the ``--add-kernel-support`` argument to the Mellanox installation script to build the kernel modules based on the version you are using. - - For diskless images, add the following line to the ``postinstall`` script: :: - - /install/postscripts/mlnxofed_ib_install \ - -p /install// -m --add-kernel-support -end- \ - -i $1 -n genimage - Preventing upgrade of the Mellanox Drivers ------------------------------------------