From 642059b3c247f7d26d3e49daa031a087b7c55792 Mon Sep 17 00:00:00 2001 From: hu-weihua Date: Tue, 12 Apr 2016 03:18:24 -0400 Subject: [PATCH 1/4] modify more content for bug fixing --- .../advanced/networks/infiniband/index.rst | 4 +- .../mlnxofed_ib_install_v2_diskful.rst | 200 +++++++++++------ .../mlnxofed_ib_install_v2_diskless.rst | 212 ++++++++++-------- .../mlnxofed_ib_install_v2_preparation.rst | 15 +- .../mlnxofed_ib_install_v2_usage.rst | 2 + .../infiniband/mlnxofed_ib_known_issue.rst | 19 ++ .../mlnxofed_ib_verified_scenario_matrix.rst | 15 ++ 7 files changed, 302 insertions(+), 165 deletions(-) create mode 100644 docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst create mode 100644 docs/source/advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix.rst diff --git a/docs/source/advanced/networks/infiniband/index.rst b/docs/source/advanced/networks/infiniband/index.rst index 1ef66d623..928ab7c80 100644 --- a/docs/source/advanced/networks/infiniband/index.rst +++ b/docs/source/advanced/networks/infiniband/index.rst @@ -1,8 +1,8 @@ Infiniband (Mellanox) ===================== -xCAT offers a certain degree support for Mellanox infiniband product, it help you to configurate Mellanox infiniband products easily. - +xCAT offers a certain degree support for Mellanox infiniband product, it help you to configurate Mellanox infiniband products easily. For more information about Mellanox infiniband, please refer to `Mellanox official site `_. + .. toctree:: :maxdepth: 2 diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst index b1eee7fff..b71ac5d37 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst @@ -1,95 +1,151 @@ Configuration for Diskful Installation ======================================= -1. Set script ``mlnxofed_ib_install`` as postbootscript :: +1. Set script ``mlnxofed_ib_install`` as ``postbootscripts`` or ``postscripts`` :: chdef -p postbootscripts="mlnxofed_ib_install -p /install//" + + Or :: + + chdef -p postscripts="mlnxofed_ib_install -p /install//" + + xCAT simulates completely the way Mellanox scripts work by using ``postbootscripts``. This way need to reboot after drive installation to make Mellanox drives work reliably just like Mellanox suggested. If you want to use the reboot after operating system installation to avoid reboot twice, you can using ``postscripts`` attribute to install Mellanox dreives. This way has been verified in limited scenarios. For more information please refer to :doc:`The Scenarioes Have Been Verified `. You can try this way in other else scenarios if you needed. -2. Specify dependence package **[required for RHEL and SLES]** +2. Specify dependency package - a) Copy a correct pkglist file **shipped by xCAT** according your environment to the ``/install/custom/install//`` directory, these pkglist files are located under ``/opt/xcat/share/xcat/install//`` :: + Some dependencies need to be installed before running Mellanox scripts. These dependencies are different between different scenario. xCAT configurates these dependency packages by using ``pkglist`` attribute of ``osimage`` definition. Please refer to :doc:`Add Additional Software Packages ` for more information:: - cp /opt/xcat/share/xcat/install//compute...pkglist \ - /install/custom/install//compute...pkglist + # lsdef -t osimage --install-compute + Object name: --install-compute + imagetype=linux + .... + pkgdir=/ + pkglist=//compute..pkglist + .... - b) Edit your ``/install/custom/install//compute...pkglist`` and add one line - - ``#INCLUDE:/opt/xcat/share/xcat/ib/netboot//ib...pkglist#`` - - You can check directory ``/opt/xcat/share/xcat/ib/netboot//`` and choose one correct ``ib...pkglist`` according your environment. - - - c) Make the related osimage use the customized pkglist :: + You can append the ib dependency packages list in the end of ``//compute..pkglist`` directly like below: :: - chdef -t osimage -o --install-compute \ - pkglist=/install/custom/install//compute...pkglist + #cat //compute..pkglist + @base + @x11 + openssl + ntp + rsyn + #ib part + createrepo + kernel-devel + kernel-source + .... + + + Or if you want to isolate IB dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute..pkglist`` like below way: :: + + #cat //compute..pkglist + @base + @x11 + openssl + ntp + rsyn + #INCLUDE://# + + xCAT has shipped some ib pkglist files under ``/opt/xcat/share/xcat/ib/netboot//``, these pkglist files have been verified in sepecific scenarion. Please refer to :doc:`The Scenarioes Have Been Verified ` to judge if you can use it directly in your environment. If so, you can use it like below: :: + + #cat //compute..pkglist + @base + @x11 + openssl + ntp + rsyn + #INCLUDE:/opt/xcat/share/xcat/ib/netboot//ib...pkglist# + + Take rhels7.2 on ppc64le for example: :: + + # lsdef -t osimage rhels7.2-ppc64le-install-compute + Object name: rhels7.2-ppc64le-install-compute + imagetype=linux + osarch=ppc64le + osdistroname=rhels7.2-ppc64le + osname=Linux + osvers=rhels7.2 + otherpkgdir=/install/post/otherpkgs/rhels7.2/ppc64le + pkgdir=/install/rhels7.2/ppc64le + pkglist=/install/custom/install/rh/compute.rhels7.ib.pkglist + profile=compute + provmethod=install + template=/opt/xcat/share/xcat/install/rh/compute.rhels7.tmpl - Take RHEL 6.4 on x86_64 for example :: - cp /opt/xcat/share/xcat/install/rh/compute.rhels6.x86_64.pkglist \ - /install/custom/install/rh/compute.rhels6.x86_64.pkglist + **[Note]**: If the osimage definition was generated by xCAT command ``copycds``, default value ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` was assigned to ``pkglist`` attribute. ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` is the sample pkglist shipped by xCAT, recommend to make a copy of this sample and using the copy in real environment. In the above example, ``/install/custom/install/rh/compute.rhels7.ib.pkglist`` is a copy of ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist``. :: + + # cat /install/custom/install/rh/compute.rhels7.ib.pkglist + #Please make sure there is a space between @ and group name + wget + ntp + nfs-utils + net-snmp + rsync + yp-tools + openssh-server + util-linux + net-tools + #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist# + + - Edit the ``/install/custom/install/rh/compute.rhels6.x86_64.pkglist`` and add below line - ``#INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels6.x86_64.pkglist#`` - - Then ``/install/custom/install/rh/compute.rhels6.x86_64.pkglist`` looks like below :: - - #Please make sure there is a space between @ and group name - #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels6.x86_64.pkglist# - ntp - nfs-utils - net-snmp - rsync - yp-tools - openssh-server - util-linux-ng - - Then modify related osimage :: - - chdef -t osimage -o rhels6.4-x86_64-install-compute \ - pkglist=/install/custom/install/rh/compute.rhels6.x86_64.pkglist - 3. Install node :: nodeset osimage=--install-compute rsetboot net rpower reset - **[Note]**: - - * In RHEL7.x, after performing all steps above, ``openibd`` doesn't work well. you can resolve this problem depending on `Mellanox OFED Linux Release Notes `_ by yourself. But reboot one more time can resolve all of these complex issues. so **we strongly recommend reboot machine again to avoid unexpected problem in RHEL7.x.** - - * If you performed firmware updates, i.e. you didn't pass ``--without-fw-update`` to option ``-m`` of ``mlnxofed_ib_install``, **reboot machine for all distro** After steps above, you can login target ndoe and find the Mellanox IB drives are located under ``/lib/modules//extra/``. - Issue ``ibstat`` command you can get the IB apater information :: + Issue ``ibv_devinfo`` command you can get the IB apater information :: + + # ibv_devinfo + hca_id: mlx5_0 + transport: InfiniBand (0) + fw_ver: 10.14.2036 + node_guid: f452:1403:0076:10e0 + sys_image_guid: f452:1403:0076:10e0 + vendor_id: 0x02c9 + vendor_part_id: 4113 + hw_ver: 0x0 + board_id: IBM1210111019 + phys_port_cnt: 2 + Device ports: + port: 1 + state: PORT_INIT (2) + max_mtu: 4096 (5) + active_mtu: 4096 (5) + sm_lid: 0 + port_lid: 65535 + port_lmc: 0x00 + link_layer: InfiniBand + + port: 2 + state: PORT_DOWN (1) + max_mtu: 4096 (5) + active_mtu: 4096 (5) + sm_lid: 0 + port_lid: 65535 + port_lmc: 0x00 + link_layer: InfiniBand + + Using ``service openibd status`` to verify if openibd works well. Below is the output in SLES :: + + #service openibd status + openibd.service - openibd - configure Mellanox devices + Loaded: loaded (/usr/lib/systemd/system/openibd.service; enabled) + Active: active (exited) since Sun 2016-04-10 07:13:46 EDT; 3h 24min ago + Docs: file:/etc/infiniband/openib.conf + Main PID: 48133 (code=exited, status=0/SUCCESS) + CGroup: /system.slice/openibd.service + + Apr 10 07:13:39 c910f05c33 openibd[48133]: [34B blob data] + Apr 10 07:13:46 c910f05c33 openibd[48133]: [49B blob data] + Apr 10 07:14:06 c910f05c33 logger[49644]: openibd: Set node_desc for mlx5_0: HCA-1 + + - [root@server ~]# ibstat - CA 'mlx4_0' - CA type: MT4099 - Number of ports: 2 - Firmware version: 2.11.500 - Hardware version: 0 - Node GUID: 0x5cf3fc000004ec02 - System image GUID: 0x5cf3fc000004ec05 - Port 1: - State: Initializing - Physical state: LinkUp - Rate: 40 (FDR10) - Base lid: 0 - LMC: 0 - SM lid: 0 - Capability mask: 0x02594868 - Port GUID: 0x5cf3fc000004ec03 - Link layer: InfiniBand - Port 2: - State: Down - Physical state: Disabled - Rate: 10 - Base lid: 0 - LMC: 0 - SM lid: 0 - Capability mask: 0x02594868 - Port GUID: 0x5cf3fc000004ec04 - Link layer: InfiniBand diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst index 37fae36c2..ccdf3005e 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst @@ -1,128 +1,160 @@ Configuration for Diskless Installation ======================================= -1. Specify dependence package **[required for RHEL and SLES]** +1. Specify dependency package - a) Copy a correct pkglist file **shipped by xCAT** according your environment to the ``/install/custom/netboot//`` directory :: + Some dependencies need to be installed before running Mellanox scripts. These dependencies are different between different scenario. xCAT configurates these dependency packages by using ``pkglist`` attribute of ``osimage`` definition. Please refer to :doc:`Add Additional Software Packages ` for more information:: - cp /opt/xcat/share/xcat/netboot//compute...pkglist \ - /install/custom/netboot//compute...pkglist + # lsdef -t osimage --netboot-compute + Object name: --netboot-compute + imagetype=linux + .... + pkgdir=/ + pkglist=//compute..pkglist + .... - b) Edit ``/install/custom/netboot//.pkglist`` and add ``#INCLUDE:/opt/xcat/share/xcat/ib/netboot//ib...pkglist#`` + You can append the ib dependency packages list in the end of ``//compute..pkglist`` directly like below: :: - For example, on RHEL 6.4 (x86_64): :: + #cat //compute..pkglist + @base + @x11 + openssl + ntp + rsyn + #ib part + createrepo + kernel-devel + kernel-source + .... - cp /opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.pkglist \ - /install/custom/netboot/rh/compute.rhels6.x86_64.pkglist - - Edit ``/install/custom/netboot/rh/compute.rhels6.x86_64.pkglist`` and add ``#INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels6.x86_64.pkglist#`` - - Then ``/install/custom/netboot/rh/compute.rhels6.x86_64.pkglist`` looks like below :: - #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels6.x86_64.pkglist# - bash - nfs-utils - openssl - dhclient - ..... + Or if you want to isolate IB dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute..pkglist`` like below way: :: + + #cat //compute..pkglist + @base + @x11 + openssl + ntp + rsyn + #INCLUDE://# + + xCAT ships some ib pkglist files under ``/opt/xcat/share/xcat/ib/netboot//``, these pkglist files have been verified in sepecific scenarion. Please refer to :doc:`The Scenarioes Have Been Verified ` to judge if you can use it directly in your environment. If so, you can use it like below: :: + + #cat //compute..pkglist + @base + @x11 + openssl + ntp + rsyn + #INCLUDE:/opt/xcat/share/xcat/ib/netboot//ib...pkglist# + 2. Prepare postinstall scripts - a) Specify a correct postinstall script **shipped by xCAT** :: - - mkdir -p /install/custom/netboot// - - cp /opt/xcat/share/xcat/netboot//.postinstall \ - /install/custom/netboot// - - chmod +x /install/custom/netboot//.postinstall + Edit ``postinstall`` script to trigger IB drvices installation during ``genimage``. Using below command to find out where the ``postinstall`` script is saved. :: + + # lsdef -t osimage --netboot-compute + Object name: --netboot-compute + .... + postinstall=/..postinstall + .... + - Take RHEL 6.4 on x86_64 for example :: - - mkdir -p /install/custom/netboot/rh/ - cp /opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.postinstall \ - /install/custom/netboot/rh/ - chmod +x /install/custom/netboot/rh/compute.rhels6.x86_64.postinstall - b) Edit ``/install/custom/netboot//.postinstall`` and add below line in the end :: + Edit ``/..postinstall`` and add below line in the end :: /install/postscripts/mlnxofed_ib_install \ -p /install// -i $1 -n genimage - **[Note]** If you want to customized kernel version (i.e the kernel version of the diskless image you want to generate is different with the kernel version of you management node), you need to pass ``--add-kernel-support`` attribute to Mellanox. the line added into ``.postinstall`` should like below :: + **[Note]** Mellanox OFED ISO was built on a certain kernal version, If your kernel version does not match with any of the Mellanox offered pre-built RPMs, you can pass ``--add-kernel-support`` attribute to Mellanox to rebuild these RPMs base on your kernel. The line added into ``.postinstall`` should like below :: /install/postscripts/mlnxofed_ib_install \ -p /install// -m --add-kernel-support -end- -i $1 -n genimage - Below steps maybe helpful for you to do judgment if you belong to this situation. - - Get the kernel version of your management node :: - - uname -r - - Get the kernel version of target image. take generating a diskless image of rhels7.0 on x86_64 for example :: - - [root@server]# lsdef -t osimage rhels7.0-x86_64-install-compute -i pkgdir - Object name: rhels7.0-x86_64-install-compute - pkgdir=/install/rhels7.0/x86_64 - - [root@server]# ls -l /install/rhels7.0/x86_64/Packages/ |grep kernel* - ....... - -r--r--r-- 1 root root 30264588 May 5 2014 kernel-3.10.0-123.el7.x86_64.rpm - ....... -3. Set the related osimage using the customized pkglist and compute.postinsall + Take rhels7.2 on ppc64le for example: :: -* [RHEL/SLES] :: + # lsdef -t osimage rhels7.2-ppc64le-netboot-compute + Object name: rhels7.2-ppc64le-netboot-compute + exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.exlist + imagetype=linux + osarch=ppc64le + osdistroname=rhels7.2-ppc64le + osname=Linux + osvers=rhels7.2 + otherpkgdir=/install/post/otherpkgs/rhels7.2/ppc64le + pkgdir=/install/rhels7.2/ppc64le + pkglist=/install/custom/install/rh/compute.rhels7.ib.pkglist + postinstall=/install/custom/install/rh/compute.rhels7.ppc64le.postinstall + profile=compute + provmethod=netboot + rootimgdir=/install/netboot/rhels7.2/ppc64le/compute - chdef -t osimage -o --netboot-compute \ - pkglist=/install/custom/netboot//compute...pkglist \ - postinstall=/install/custom/netboot//.postinstall -* [Ubuntu] :: + **[Note]**: If the osimage definition was generated by xCAT command ``copycds``, default value ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` was assigned to ``pkglist`` attribute. ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` is the sample pkglist shipped by xCAT, recommend to make a copy of this sample and using the copy in real environment. In the above example, ``/install/custom/install/rh/compute.rhels7.ib.pkglist`` is a copy of ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist``. For the same reason, ``/install/custom/install/rh/compute.rhels7.ppc64le.postinstall`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall``. :: - chdef -t osimage -o --netboot-compute \ - postinstall=/install/custom/netboot//.postinstall + # cat /install/custom/install/rh/compute.rhels7.ib.pkglist + #Please make sure there is a space between @ and group name + wget + ntp + nfs-utils + net-snmp + rsync + yp-tools + openssh-server + util-linux + net-tools + #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist# -4. Generate and package image for diskless installation :: + + #cat /install/custom/install/rh/compute.rhels7.ppc64le.postinstall + #!/bin/sh + #-- Do not remove following line if you want to make use of CVS version tracking + ...... + # [ -r $workdir/$profile.$ext ] && cat $workdir/$profile.$ext | grep -E '^[[:space:]]*#.*[[:space:]]\$Id' >> $installroot/etc/IMGVERSION + #done + /install/postscripts/mlnxofed_ib_install -p /install/ofed/MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso -i $1 -n genimage + +3. Generate and package image for diskless installation :: genimage --netboot-compute packimage --netboot-compute -5. Install node :: +4. Install node :: nodeset osimage=--netboot-compute rsetboot net rpower reset - After installation, you can login target ndoe and issue ``ibstat`` command to verify if your IB driver works well. if everything is fine, you can get the IB apater information :: - - [root@server ~]# ibstat - CA 'mlx4_0' - CA type: MT4099 - Number of ports: 2 - Firmware version: 2.11.500 - Hardware version: 0 - Node GUID: 0x5cf3fc000004ec02 - System image GUID: 0x5cf3fc000004ec05 - Port 1: - State: Initializing - Physical state: LinkUp - Rate: 40 (FDR10) - Base lid: 0 - LMC: 0 - SM lid: 0 - Capability mask: 0x02594868 - Port GUID: 0x5cf3fc000004ec03 - Link layer: InfiniBand - Port 2: - State: Down - Physical state: Disabled - Rate: 10 - Base lid: 0 - LMC: 0 - SM lid: 0 - Capability mask: 0x02594868 - Port GUID: 0x5cf3fc000004ec04 - Link layer: InfiniBand + After installation, you can login target ndoe and issue ``ibv_devinfo`` command to verify if your IB driver works well. if everything is fine, you can get the IB apater information :: + + # ibv_devinfo + hca_id: mlx5_0 + transport: InfiniBand (0) + fw_ver: 10.14.2036 + node_guid: f452:1403:0076:10e0 + sys_image_guid: f452:1403:0076:10e0 + vendor_id: 0x02c9 + vendor_part_id: 4113 + hw_ver: 0x0 + board_id: IBM1210111019 + phys_port_cnt: 2 + Device ports: + port: 1 + state: PORT_INIT (2) + max_mtu: 4096 (5) + active_mtu: 4096 (5) + sm_lid: 0 + port_lid: 65535 + port_lmc: 0x00 + link_layer: InfiniBand + + port: 2 + state: PORT_DOWN (1) + max_mtu: 4096 (5) + active_mtu: 4096 (5) + sm_lid: 0 + port_lid: 65535 + port_lmc: 0x00 + link_layer: InfiniBand diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst index 217eb828d..3b26a2a9e 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst @@ -1,12 +1,22 @@ Preparation =========== -Obtain the Mellanox OFED ISO file from `Mellanox official site `_ and put it into one place under ``/install`` directory depending on your need. +Obtain the Mellanox OFED ISO +---------------------------- + +Obtain the Mellanox OFED ISO file from `Mellanox official Download Page `_ and put it into one place under ``/install`` directory depending on your need. **[NOTE]** * Mellanox provides OFED drivers in **tarball** and **iso** formats. xCAT only supports the **iso** format at this time. * Mellanox provides different OFED ISOs depending on operating system and machine architecture, named like MLNX_OFED_LINUX----.iso, you should download correct one according your environment. +* Mellanox has some updates and known issues for echo OFED, please read `InfiniBand/VPI Software Overview `_ to understand these information. +* The Mellanox links offered above maybe outdate in future for Mellanox updates his web page, xCAT will keep updating for synchronization. If we don't update in time, please access `Mellanox web portal `_ to find ``Support/Education`` then ``InfiniBand/VPI Drivers`` lables. + +Prepare Install Script +---------------------- + +**mlnxofed_ib_install.v2** is a sample script, its framework can help you install Mellanox drives easily. But in specific scenario, some detail need to be modified to meet requirement, such like dependency package list. It has been verified in limited scenarios and can work as solution in these scenarios. For these scenarions information please refer to :doc:`The Scenarioes Have Been Verified `. Copy **mlnxofed_ib_install.v2** into ``/install/postscripts`` and change name to **mlnxofed_ib_install** :: @@ -31,3 +41,6 @@ If need to pass ``--without-32bit --without-fw-update --add-kernel-support --for mlnxofed_ib_install -p /install// \ -m --without-32bit --without-fw-update --add-kernel-support --force -end- + +**[Note]** We recommend to update your firmware to the latest version to avoid unexpected problem when you install ib for the first time. + diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_usage.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_usage.rst index be8532360..52d500fb3 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_usage.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_usage.rst @@ -7,4 +7,6 @@ Using mlnxofed_ib_install.v2 (Recommend) mlnxofed_ib_install_v2_preparation.rst mlnxofed_ib_install_v2_diskful.rst mlnxofed_ib_install_v2_diskless.rst + mlnxofed_ib_verified_scenario_matrix.rst + mlnxofed_ib_known_issue.rst diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst new file mode 100644 index 000000000..ef0ee44c6 --- /dev/null +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_known_issue.rst @@ -0,0 +1,19 @@ +Known Issues +============ + + +Known Issue 1 +------------- + +After you install mellanox derives in rhels7.2 successfully by xCAT, maybe you have new requirement to upgrade your operating system to higher version. In this case you probably hit such problem the IB adaptor drives shipped by operating system is higher than the Mellanox drives you have installed. That means the the Mellanox drives will be replaced by the drives shipped by operating system. If it's not the result you expect, you hope keep the Mellanox drives after operating system upgraded, please add below statement into ``/etc/yum.conf`` in your target node after you install mellanox derives successfully for the first time. :: + + exclude=dapl* libib* ibacm infiniband* libmlx* librdma* opensm* ibutils* + + +Known Issue 2 +------------- + +If you want to use ``--add-kernel-support`` attribute in sles12.1 and ppc64le scenario, you will find some dependency packages are not shipped by SLES Server DVDs, such like ``python-devel``, it's shipped in SDK DVDs. xCAT doesn't ship specific pkglist to support such scenario. If you have such requirement, please used ``otherpkglist`` and ``otherpkgs`` attributes to prepare dependency packages repository ahead. If you need help about ``otherpkglist`` and ``otherpkgs``attributes, please refer to :doc:`Add Additional Software Packages `. + + + diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix.rst new file mode 100644 index 000000000..f1dd91130 --- /dev/null +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix.rst @@ -0,0 +1,15 @@ +The Scenarioes Have Been Verified +================================= + + ++---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ +| OS version | Arch | Ofed version | Attribute supported by mlnx | IB.pkglist | ++===============+=========+===================================================+==================================================================+===========================+ +| rhels7.1 | ppc64 | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.1-ppc64.iso |--without-32bit --without-fw-update --add-kernel-support --force | ib.rhels7.ppc64.pkglist | ++---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ +| rhels7.2 | ppc64le | MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso |--without-32bit --without-fw-update --add-kernel-support --force | ib.rhels7.ppc64le.pkglist | ++---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ +| sles12.1 | ppc64le |MLNX_OFED_LINUX-3.2-2.0.0.0-sles12sp1-ppc64le.iso |--without-32bit --without-fw-update --force | ib.sles12.ppc64le.pkglist | ++---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ +| ubuntu14.04.3 | ppc64le |MLNX_OFED_LINUX-3.2-2.0.0.0-ubuntu14.04-ppc64le.iso|--without-32bit --without-fw-update --add-kernel-support --force |ib.ubuntu14.ppc64le.pkglist| ++---------------+---------+---------------------------------------------------+------------------------------------------------------------------+---------------------------+ From b2a8ddecf072149007c0e5c3ba95b612f12133c1 Mon Sep 17 00:00:00 2001 From: hu-weihua Date: Thu, 14 Apr 2016 03:25:54 -0400 Subject: [PATCH 2/4] update example --- .../mlnxofed_ib_install_v2_diskful.rst | 60 +++++++---- .../mlnxofed_ib_install_v2_diskless.rst | 99 ++++++++++--------- 2 files changed, 93 insertions(+), 66 deletions(-) diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst index b71ac5d37..601f03229 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst @@ -3,11 +3,11 @@ Configuration for Diskful Installation 1. Set script ``mlnxofed_ib_install`` as ``postbootscripts`` or ``postscripts`` :: - chdef -p postbootscripts="mlnxofed_ib_install -p /install//" + chdef -p postbootscripts="mlnxofed_ib_install -p /install//" Or :: - chdef -p postscripts="mlnxofed_ib_install -p /install//" + chdef -p postscripts="mlnxofed_ib_install -p /install//" xCAT simulates completely the way Mellanox scripts work by using ``postbootscripts``. This way need to reboot after drive installation to make Mellanox drives work reliably just like Mellanox suggested. If you want to use the reboot after operating system installation to avoid reboot twice, you can using ``postscripts`` attribute to install Mellanox dreives. This way has been verified in limited scenarios. For more information please refer to :doc:`The Scenarioes Have Been Verified `. You can try this way in other else scenarios if you needed. @@ -20,12 +20,12 @@ Configuration for Diskful Installation imagetype=linux .... pkgdir=/ - pkglist=//compute..pkglist + pkglist=//compute...pkglist .... - You can append the ib dependency packages list in the end of ``//compute..pkglist`` directly like below: :: + You can append the ib dependency packages list in the end of ``//compute...pkglist`` directly like below: :: - #cat //compute..pkglist + #cat //compute...pkglist @base @x11 openssl @@ -38,9 +38,9 @@ Configuration for Diskful Installation .... - Or if you want to isolate IB dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute..pkglist`` like below way: :: + Or if you want to isolate IB dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute...pkglist`` like below way: :: - #cat //compute..pkglist + #cat //compute...pkglist @base @x11 openssl @@ -50,7 +50,7 @@ Configuration for Diskful Installation xCAT has shipped some ib pkglist files under ``/opt/xcat/share/xcat/ib/netboot//``, these pkglist files have been verified in sepecific scenarion. Please refer to :doc:`The Scenarioes Have Been Verified ` to judge if you can use it directly in your environment. If so, you can use it like below: :: - #cat //compute..pkglist + #cat //compute...pkglist @base @x11 openssl @@ -133,19 +133,37 @@ Configuration for Diskful Installation port_lmc: 0x00 link_layer: InfiniBand - Using ``service openibd status`` to verify if openibd works well. Below is the output in SLES :: - - #service openibd status - openibd.service - openibd - configure Mellanox devices - Loaded: loaded (/usr/lib/systemd/system/openibd.service; enabled) - Active: active (exited) since Sun 2016-04-10 07:13:46 EDT; 3h 24min ago - Docs: file:/etc/infiniband/openib.conf - Main PID: 48133 (code=exited, status=0/SUCCESS) - CGroup: /system.slice/openibd.service - - Apr 10 07:13:39 c910f05c33 openibd[48133]: [34B blob data] - Apr 10 07:13:46 c910f05c33 openibd[48133]: [49B blob data] - Apr 10 07:14:06 c910f05c33 logger[49644]: openibd: Set node_desc for mlx5_0: HCA-1 + Using ``service openibd status`` to verify if openibd works well. Below is the output in rhels7.2. :: + # service openibd status + HCA driver loaded + + Configured IPoIB devices: + ib0 ib1 + + Currently active IPoIB devices: + Configured Mellanox EN devices: + + Currently active Mellanox devices: + + The following OFED modules are loaded: + + rdma_ucm + rdma_cm + ib_addr + ib_ipoib + mlx4_core + mlx4_ib + mlx4_en + mlx5_core + mlx5_ib + ib_uverbs + ib_umad + ib_ucm + ib_sa + ib_cm + ib_mad + ib_core + diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst index ccdf3005e..3a7636abd 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst @@ -10,17 +10,19 @@ Configuration for Diskless Installation imagetype=linux .... pkgdir=/ - pkglist=//compute..pkglist + pkglist=//compute...pkglist .... - You can append the ib dependency packages list in the end of ``//compute..pkglist`` directly like below: :: + You can append the ib dependency packages list in the end of ``//compute...pkglist`` directly like below: :: - #cat //compute..pkglist - @base - @x11 + #cat //compute...pkglist + bash + nfs-utils openssl - ntp - rsyn + dhclient + kernel + ..... + #ib part createrepo kernel-devel @@ -28,24 +30,26 @@ Configuration for Diskless Installation .... - Or if you want to isolate IB dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute..pkglist`` like below way: :: + Or if you want to isolate IB dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute...pkglist`` like below way: :: - #cat //compute..pkglist - @base - @x11 + #cat //compute...pkglist + bash + nfs-utils openssl - ntp - rsyn + dhclient + kernel + ..... #INCLUDE://# xCAT ships some ib pkglist files under ``/opt/xcat/share/xcat/ib/netboot//``, these pkglist files have been verified in sepecific scenarion. Please refer to :doc:`The Scenarioes Have Been Verified ` to judge if you can use it directly in your environment. If so, you can use it like below: :: - #cat //compute..pkglist - @base - @x11 + #cat //compute...pkglist + bash + nfs-utils openssl - ntp - rsyn + dhclient + kernel + ..... #INCLUDE:/opt/xcat/share/xcat/ib/netboot//ib...pkglist# @@ -70,48 +74,53 @@ Configuration for Diskless Installation **[Note]** Mellanox OFED ISO was built on a certain kernal version, If your kernel version does not match with any of the Mellanox offered pre-built RPMs, you can pass ``--add-kernel-support`` attribute to Mellanox to rebuild these RPMs base on your kernel. The line added into ``.postinstall`` should like below :: /install/postscripts/mlnxofed_ib_install \ - -p /install// -m --add-kernel-support -end- -i $1 -n genimage + -p /install// -m --add-kernel-support -end- -i $1 -n genimage Take rhels7.2 on ppc64le for example: :: - # lsdef -t osimage rhels7.2-ppc64le-netboot-compute + #lsdef -t osimage rhels7.2-ppc64le-netboot-compute Object name: rhels7.2-ppc64le-netboot-compute - exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.exlist - imagetype=linux - osarch=ppc64le - osdistroname=rhels7.2-ppc64le - osname=Linux - osvers=rhels7.2 - otherpkgdir=/install/post/otherpkgs/rhels7.2/ppc64le - pkgdir=/install/rhels7.2/ppc64le - pkglist=/install/custom/install/rh/compute.rhels7.ib.pkglist - postinstall=/install/custom/install/rh/compute.rhels7.ppc64le.postinstall - profile=compute - provmethod=netboot - rootimgdir=/install/netboot/rhels7.2/ppc64le/compute + exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.exlist + imagetype=linux + osarch=ppc64le + osdistroname=rhels7.2-ppc64le + osname=Linux + osvers=rhels7.2 + otherpkgdir=/install/post/otherpkgs/rhels7.2/ppc64le + permission=755 + pkgdir=/install/rhels7.2/ppc64le + pkglist=/install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist + postinstall=/install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall + profile=compute + provmethod=netboot + rootimgdir=/install/netboot/rhels7.2/ppc64le/compute - **[Note]**: If the osimage definition was generated by xCAT command ``copycds``, default value ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` was assigned to ``pkglist`` attribute. ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist`` is the sample pkglist shipped by xCAT, recommend to make a copy of this sample and using the copy in real environment. In the above example, ``/install/custom/install/rh/compute.rhels7.ib.pkglist`` is a copy of ``/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist``. For the same reason, ``/install/custom/install/rh/compute.rhels7.ppc64le.postinstall`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall``. :: + **[Note]**: If the osimage definition was generated by xCAT command ``copycds``, default value ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist`` was assigned to ``pkglist`` attribute. ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist`` is the sample pkglist shipped by xCAT, recommend to make a copy of this sample and using the copy in real environment. In the above example, ``/install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist``. For the same reason, ``/install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall``. :: - # cat /install/custom/install/rh/compute.rhels7.ib.pkglist - #Please make sure there is a space between @ and group name - wget - ntp + ``compute.rhels7.ppc64le.pkglist`` looks like below: :: + + # cat /install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist + bash nfs-utils - net-snmp - rsync - yp-tools - openssh-server - util-linux + openssl + dhclient + bc + ...... + lsvpd + irqbalance + procps-ng + parted net-tools #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist# + ``compute.rhels7.ppc64le.ib.postinstall`` looks like below: :: - #cat /install/custom/install/rh/compute.rhels7.ppc64le.postinstall + # cat /install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall #!/bin/sh #-- Do not remove following line if you want to make use of CVS version tracking - ...... + ..... # [ -r $workdir/$profile.$ext ] && cat $workdir/$profile.$ext | grep -E '^[[:space:]]*#.*[[:space:]]\$Id' >> $installroot/etc/IMGVERSION #done /install/postscripts/mlnxofed_ib_install -p /install/ofed/MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso -i $1 -n genimage From 9ffa8e314deea7a19f127fae0e1eb697dc25d47b Mon Sep 17 00:00:00 2001 From: hu-weihua Date: Fri, 15 Apr 2016 03:44:38 -0400 Subject: [PATCH 3/4] modify grammar error dependig on gongjie's comments --- .../infiniband/mlnxofed_ib_install_v2_diskful.rst | 6 +++--- .../infiniband/mlnxofed_ib_install_v2_diskless.rst | 14 +++++++------- .../mlnxofed_ib_install_v2_preparation.rst | 8 ++++---- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst index 601f03229..78f5a35d5 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst @@ -38,7 +38,7 @@ Configuration for Diskful Installation .... - Or if you want to isolate IB dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute...pkglist`` like below way: :: + Or if you want to isolate InfiniBand dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute...pkglist`` like below way: :: #cat //compute...pkglist @base @@ -99,9 +99,9 @@ Configuration for Diskful Installation rpower reset - After steps above, you can login target ndoe and find the Mellanox IB drives are located under ``/lib/modules//extra/``. + After steps above, you can login target node and find the Mellanox InfiniBand drives are located under ``/lib/modules//extra/``. - Issue ``ibv_devinfo`` command you can get the IB apater information :: + Issue ``ibv_devinfo`` command you can get the InfiniBand apater information :: # ibv_devinfo hca_id: mlx5_0 diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst index 3a7636abd..9e83bc452 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskless.rst @@ -3,7 +3,7 @@ Configuration for Diskless Installation 1. Specify dependency package - Some dependencies need to be installed before running Mellanox scripts. These dependencies are different between different scenario. xCAT configurates these dependency packages by using ``pkglist`` attribute of ``osimage`` definition. Please refer to :doc:`Add Additional Software Packages ` for more information:: + Some dependencies need to be installed before running Mellanox scripts. These dependencies are different among different scenarios. xCAT can help user to install these dependency packages by adding these package names to the file specified by the ``pkglist`` attribute of the ``osimage`` definition. Please refer to :doc:`Add Additional Software Packages ` for more information:: # lsdef -t osimage --netboot-compute Object name: --netboot-compute @@ -30,7 +30,7 @@ Configuration for Diskless Installation .... - Or if you want to isolate IB dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute...pkglist`` like below way: :: + Or if you want to isolate InfiniBand dependency packages list into a separate file, after you edit this file, you can append the file in ``//compute...pkglist`` like below way: :: #cat //compute...pkglist bash @@ -41,7 +41,7 @@ Configuration for Diskless Installation ..... #INCLUDE://# - xCAT ships some ib pkglist files under ``/opt/xcat/share/xcat/ib/netboot//``, these pkglist files have been verified in sepecific scenarion. Please refer to :doc:`The Scenarioes Have Been Verified ` to judge if you can use it directly in your environment. If so, you can use it like below: :: + xCAT ships some InfiniBand pkglist files under ``/opt/xcat/share/xcat/ib/netboot//``, these pkglist files have been verified in sepecific scenarion. Please refer to :doc:`The Scenarioes Have Been Verified ` to judge if you can use it directly in your environment. If so, you can use it like below: :: #cat //compute...pkglist bash @@ -55,7 +55,7 @@ Configuration for Diskless Installation 2. Prepare postinstall scripts - Edit ``postinstall`` script to trigger IB drvices installation during ``genimage``. Using below command to find out where the ``postinstall`` script is saved. :: + Edit ``postinstall`` script to trigger InfniBand drvices installation during ``genimage``. Using below command to find out where the ``postinstall`` script is defined. :: # lsdef -t osimage --netboot-compute Object name: --netboot-compute @@ -71,8 +71,8 @@ Configuration for Diskless Installation -p /install// -i $1 -n genimage - **[Note]** Mellanox OFED ISO was built on a certain kernal version, If your kernel version does not match with any of the Mellanox offered pre-built RPMs, you can pass ``--add-kernel-support`` attribute to Mellanox to rebuild these RPMs base on your kernel. The line added into ``.postinstall`` should like below :: - + **[Note]** Mellanox OFED ISO was built against a series of certain kernael versions, If the version of linux kernel you are using does not match with any of the Mellanox offered pre-built kernel modules, you can pass ``--add-kernel-support`` command line argument to Mellanox OFED installation script to build these kernel modules base on the version of linux kernel you are using. The line added into ``.postinstall`` should like below :: + /install/postscripts/mlnxofed_ib_install \ -p /install// -m --add-kernel-support -end- -i $1 -n genimage @@ -136,7 +136,7 @@ Configuration for Diskless Installation rsetboot net rpower reset - After installation, you can login target ndoe and issue ``ibv_devinfo`` command to verify if your IB driver works well. if everything is fine, you can get the IB apater information :: + After installation, you can login target ndoe and issue ``ibv_devinfo`` command to verify if your InfiniBand driver works well. if everything is fine, you can get the InfiniBand apater information :: # ibv_devinfo hca_id: mlx5_0 diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst index 3b26a2a9e..d83840b4e 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_preparation.rst @@ -8,10 +8,10 @@ Obtain the Mellanox OFED ISO file from `Mellanox official Download Page ---.iso, you should download correct one according your environment. +* Mellanox provides OFED drivers in **tarball** and **ISO image** formats. xCAT only supports the **iso** format at this time. +* Mellanox provides different OFED ISOs depending on operating system and machine architecture, named like MLNX_OFED_LINUX----.iso, you should download correct one according to your environment. * Mellanox has some updates and known issues for echo OFED, please read `InfiniBand/VPI Software Overview `_ to understand these information. -* The Mellanox links offered above maybe outdate in future for Mellanox updates his web page, xCAT will keep updating for synchronization. If we don't update in time, please access `Mellanox web portal `_ to find ``Support/Education`` then ``InfiniBand/VPI Drivers`` lables. +* The Mellanox links offered above maybe outdate in future for Mellanox updates its web page, xCAT will keep updating for synchronization. If we don't update in time, please access `Mellanox web portal `_ to find ``Support/Education`` then ``InfiniBand/VPI Drivers`` lables. Prepare Install Script ---------------------- @@ -42,5 +42,5 @@ If need to pass ``--without-32bit --without-fw-update --add-kernel-support --for mlnxofed_ib_install -p /install// \ -m --without-32bit --without-fw-update --add-kernel-support --force -end- -**[Note]** We recommend to update your firmware to the latest version to avoid unexpected problem when you install ib for the first time. +**[Note]** We recommend to update your firmware to the version Mellanox supported in its release notes to avoid unexpected problem when you install InfiniBand driver. From 49a85041fe685a8a13c6faa11adf0c2422fd9bba Mon Sep 17 00:00:00 2001 From: hu-weihua Date: Fri, 15 Apr 2016 04:07:25 -0400 Subject: [PATCH 4/4] replace infiniband to InfiniBand --- .../advanced/networks/infiniband/driver_and_installation.rst | 4 ++-- docs/source/advanced/networks/infiniband/index.rst | 4 ++-- .../networks/infiniband/mlnxofed_ib_install_v2_diskful.rst | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/advanced/networks/infiniband/driver_and_installation.rst b/docs/source/advanced/networks/infiniband/driver_and_installation.rst index 1c22dfed9..d9d10a596 100644 --- a/docs/source/advanced/networks/infiniband/driver_and_installation.rst +++ b/docs/source/advanced/networks/infiniband/driver_and_installation.rst @@ -1,11 +1,11 @@ IB Driver Preparation and Installation ====================================== -xCAT provides sample postscripts to help you install the Mellanox OpenFabrics Enterprise Distribution (OFED) Infiniband Driver. These scripts are located in ``opt/xcat/share/xcat/ib/scripts/Mellanox/``. You can use these scripts directly or change them to satisfy your own environment. **xCAT 2.11 drops support of mlnxofed_ib_install and recommends using version 2 of the script: mlnxofed_ib_install.v2**. +xCAT provides sample postscripts to help you install the Mellanox OpenFabrics Enterprise Distribution (OFED) InfiniBand Driver. These scripts are located in ``opt/xcat/share/xcat/ib/scripts/Mellanox/``. You can use these scripts directly or change them to satisfy your own environment. **xCAT 2.11 drops support of mlnxofed_ib_install and recommends using version 2 of the script: mlnxofed_ib_install.v2**. .. toctree:: :maxdepth: 2 mlnxofed_ib_install_v2_usage.rst mlnxofed_ib_install_v1_usage.rst - \ No newline at end of file + diff --git a/docs/source/advanced/networks/infiniband/index.rst b/docs/source/advanced/networks/infiniband/index.rst index 928ab7c80..c525d6c95 100644 --- a/docs/source/advanced/networks/infiniband/index.rst +++ b/docs/source/advanced/networks/infiniband/index.rst @@ -1,7 +1,7 @@ -Infiniband (Mellanox) +InfiniBand (Mellanox) ===================== -xCAT offers a certain degree support for Mellanox infiniband product, it help you to configurate Mellanox infiniband products easily. For more information about Mellanox infiniband, please refer to `Mellanox official site `_. +xCAT offers a certain degree support for Mellanox InfiniBand product, it help you to configurate Mellanox InfiniBand products easily. For more information about Mellanox InfiniBand, please refer to `Mellanox official site `_. .. toctree:: :maxdepth: 2 diff --git a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst index 78f5a35d5..449009446 100644 --- a/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst +++ b/docs/source/advanced/networks/infiniband/mlnxofed_ib_install_v2_diskful.rst @@ -9,7 +9,7 @@ Configuration for Diskful Installation chdef -p postscripts="mlnxofed_ib_install -p /install//" - xCAT simulates completely the way Mellanox scripts work by using ``postbootscripts``. This way need to reboot after drive installation to make Mellanox drives work reliably just like Mellanox suggested. If you want to use the reboot after operating system installation to avoid reboot twice, you can using ``postscripts`` attribute to install Mellanox dreives. This way has been verified in limited scenarios. For more information please refer to :doc:`The Scenarioes Have Been Verified `. You can try this way in other else scenarios if you needed. + xCAT simulates completely the way Mellanox scripts work by using ``postbootscripts``. This way need to reboot after drive installation to make Mellanox drivers work reliably just like Mellanox suggested. If you want to use the reboot after operating system installation to avoid reboot twice, you can using ``postscripts`` attribute to install Mellanox drivers. This way has been verified in limited scenarios. For more information please refer to :doc:`The Scenarioes Have Been Verified `. You can try this way in other else scenarios if you needed. 2. Specify dependency package