Configuration for Diskless Installation
=======================================

1. Specify dependency package 

  Some dependencies need to be installed before running Mellanox scripts. These dependencies are different among different scenarios. xCAT can help user to install these dependency packages by adding these package names to the file specified by the ``pkglist`` attribute of the ``osimage`` definition. Please refer to :doc:`Add Additional Software Packages </guides/admin-guides/manage_clusters/ppc64le/diskful/customize_image/additional_pkg>` for more information::

    # lsdef -t osimage  <osver>-<arch>-netboot-compute 
    Object name:   <osver>-<arch>-netboot-compute
    imagetype=linux
    ....
    pkgdir=/<os packages directory>
    pkglist=/<os packages list directory>/compute.<os>.<arch>.pkglist
    ....

  You can append the ib dependency packages list in the end of ``/<os packages list directory>/compute.<os>.<arch>.pkglist`` directly like below: ::

    #cat /<os packages list directory>/compute.<os>.<arch>.pkglist
    bash
    nfs-utils
    openssl
    dhclient
    kernel
    .....

    #ib part
    createrepo
    kernel-devel
    kernel-source
    ....


  Or if you want to isolate InfiniBand dependency packages list into a separate file, after you edit this file, you can append the file in ``/<os packages list directory>/compute.<os>.<arch>.pkglist`` like below way: ::

    #cat /<os packages list directory>/compute.<os>.<arch>.pkglist
    bash
    nfs-utils
    openssl
    dhclient
    kernel
    .....
    #INCLUDE:/<ib pkglist path>/<you ib pkglist file>#

  xCAT ships some InfiniBand pkglist files under ``/opt/xcat/share/xcat/ib/netboot/<ostype>/``, these pkglist files have been verified in sepecific scenario. Please refer to :doc:`The Scenarios Have Been Verified </advanced/networks/infiniband/mlnxofed_ib_verified_scenario_matrix>` to judge if you can use it directly in your environment. If so, you can use it like below: ::

    #cat /<os packages list directory>/compute.<os>.<arch>.pkglist
    bash
    nfs-utils
    openssl
    dhclient
    kernel
    .....
    #INCLUDE:/opt/xcat/share/xcat/ib/netboot/<ostype>/ib.<os>.<arch>.pkglist#


2. Prepare postinstall scripts 

  Edit ``postinstall`` script to trigger InfniBand drvices installation during ``genimage``. Using below command to find out where the ``postinstall`` script is defined. ::
 
    # lsdef -t osimage <os>-<arch>-netboot-compute
    Object name: <os>-<arch>-netboot-compute
    ....
    postinstall=/<postinstall script path/compute.<os>.<arch>.postinstall
    ....


		
  Edit ``/<postinstall script path/compute.<os>.<arch>.postinstall`` and add below line in the end. ::

        /install/postscripts/mlnxofed_ib_install \
        -p /install/<path>/<MLNX_OFED_LINUX.iso> -i $1 -n genimage


 **[Note]** Mellanox OFED ISO was built against a series of certain kernael versions, If the version of linux kernel you are using does not match with any of the Mellanox offered pre-built kernel modules, you can pass ``--add-kernel-support`` command line argument to Mellanox OFED installation script to build these kernel modules base on the version of linux kernel you are using. The line added into ``<profile>.postinstall`` should like below. :: 

        /install/postscripts/mlnxofed_ib_install \
        -p /install/<subpath>/<MLNX_OFED_LINUX.iso> -m --add-kernel-support -end- -i $1 -n genimage
  
		
 Take rhels7.2 on ppc64le for example:  ::

    #lsdef -t osimage  rhels7.2-ppc64le-netboot-compute
    Object name: rhels7.2-ppc64le-netboot-compute
        exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.exlist
        imagetype=linux
        osarch=ppc64le
        osdistroname=rhels7.2-ppc64le
        osname=Linux
        osvers=rhels7.2
        otherpkgdir=/install/post/otherpkgs/rhels7.2/ppc64le
        permission=755
        pkgdir=/install/rhels7.2/ppc64le
        pkglist=/install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist
        postinstall=/install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall
        profile=compute
        provmethod=netboot
        rootimgdir=/install/netboot/rhels7.2/ppc64le/compute


 **[Note]**: If the osimage definition was generated by xCAT command ``copycds``, default value ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist`` was assigned to ``pkglist`` attribute. ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist`` is the sample pkglist shipped by xCAT, recommend to make a copy of this sample and using the copy in real environment. In the above example, ``/install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist``. For the same reason, ``/install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall`` is a copy of ``/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall``.

 ``compute.rhels7.ppc64le.pkglist`` looks like below:  ::

    # cat /install/custom/netboot/rh/compute.rhels7.ppc64le.pkglist
    bash
    nfs-utils
    openssl
    dhclient
    bc
    ......
    lsvpd
    irqbalance
    procps-ng
    parted
    net-tools
    #INCLUDE:/opt/xcat/share/xcat/ib/netboot/rh/ib.rhels7.ppc64le.pkglist#

 ``compute.rhels7.ppc64le.ib.postinstall`` looks like below: ::

    # cat /install/custom/netboot/rh/compute.rhels7.ppc64le.ib.postinstall
    #!/bin/sh
    #-- Do not remove following line if you want to make use of CVS version tracking
    .....
    #  [ -r $workdir/$profile.$ext ] && cat $workdir/$profile.$ext | grep -E '^[[:space:]]*#.*[[:space:]]\$Id' >> $installroot/etc/IMGVERSION
    #done
    /install/postscripts/mlnxofed_ib_install -p /install/ofed/MLNX_OFED_LINUX-3.2-2.0.0.0-rhel7.2-ppc64le.iso -i $1 -n genimage   

3. Generate and package image for diskless installation ::

	genimage   <osver>-<arch>-netboot-compute 
	packimage  <osver>-<arch>-netboot-compute

4. Install node ::

	nodeset <nodename> osimage=<osver>-<arch>-netboot-compute 
	rsetboot <nodename> net
	rpower <nodename> reset

  After installation, you can login target ndoe and issue ``ibv_devinfo`` command to verify if your InfiniBand driver works well. if everything is fine, you can get the InfiniBand apater information. ::

    # ibv_devinfo
    hca_id:	mlx5_0
	transport:			InfiniBand (0)
	fw_ver:				10.14.2036
	node_guid:			f452:1403:0076:10e0
	sys_image_guid:			f452:1403:0076:10e0
	vendor_id:			0x02c9
	vendor_part_id:			4113
	hw_ver:				0x0
	board_id:			IBM1210111019
	phys_port_cnt:			2
	Device ports:
		port:	1
			state:			PORT_INIT (2)
			max_mtu:		4096 (5)
			active_mtu:		4096 (5)
			sm_lid:			0
			port_lid:		65535
			port_lmc:		0x00
			link_layer:		InfiniBand

		port:	2
			state:			PORT_DOWN (1)
			max_mtu:		4096 (5)
			active_mtu:		4096 (5)
			sm_lid:			0
			port_lid:		65535
			port_lmc:		0x00
			link_layer:		InfiniBand	
