# IBM(c) 2008 EPL license http://www.eclipse.org/legal/epl-v10.html

configiba.README

This README describes how to use the configiba script.

This configiba postscript is used to do secondary adapter configuration for InfiniBand interfaces (ibX) and ml0 on both AIX and Linux Managed Nodes.  If only one port on each InfiniBand adapter is available, please copy configiba.1port to /install/postscripts.  If both ports on each adapter are available, please copy configiba.2ports to /install/postscripts.

IB configuration with xCAT needs nameserver support to provide IP addresses for computes nodes.  We assume nameserver is setup on Management node.  Please follow similiar steps on the system that provide nameserver service if nameserver is not on management node.

There are two times to configure IB adapters, during node installation or with command updatenode after node installation.  These two IB configuration ways are similar and finally they will trigger this sample postscript to configure IB adapters.  

To use this script, users need to firstly setup DNS for the new adapters before node installation or before triggering command updatenode depending on when to start IB configuration, either time is supported: 

1.The IP address entries for IB interfaces in /etc/hosts on xCAT management nodes should have the node short hostname and the unique IB interface name in them.  The format should be <ip_address_for_this_ib_interface node_short_hostname-ib_interfacename>.
For example:
c890f11ec01 is the node short hostname, c890f11ec01-ib0, c890f11ec01-ib1, c890f11ec01-ib2, etc. are the IP names for the IB interfaces on c890f11ec01.

2.Update networks table with IB sub-network
For example:
chtab net=172.16.0.0 networks.netname=ib0 networks.mask=255.255.0.0 networks.mgtifname=ib
Note: Attributes gateway, dhcpserver, tftpserver, and nameservers in networks table are not a MUST to be assigned, since the xCAT management work is still running on ethernet.

3.Add the entries in the /etc/hosts into DNS and restart the DNS
For Linux Managed Nodes:
makedns
service named restart
For AIX Managed Nodes:
makedns
stopsrc -s named 
startsrc -s named
lssrc -s named 
Note: Only if the state of named is active, we can continue to test DNS.

4.Check if DNS for the IB network has been setup successfully
nslookup [node_short_hostname-ib_interfacename]

5.Update /etc/resolv.conf
If the computer node have already been installed and are runnning, make sure /etc/resolv.conf is available on the compute node before running updatenode, since configiba script will connect to name server to resolve IP address for the IB interfaces. If not, define /etc/resolv.conf on compute node or use rcp to copy resolv.conf from management node to the compute node. Following is an example of /etc/resolv.conf:
nameserver 172.16.0.13
domain ppd.pok.ibm.com
search ppd.pok.ibm.com

Note: 172.16.0.13 is the name server address which could provide the
IP addresses for IB interfaces on compute nodes.

6.Add this script to postscripts
chtab node=c890f11ec01 postscripts.postscripts=configiba.1port
or 
chtab node=c890f11ec01 postscripts.postscripts=configiba.2ports
Note: The postscripts name depends on the script copied to /install/postscripts/

7.Now all the preparation work for IB configuration has been done, user can continue the general node installation or trigger updatenode command to configure IB adapters.

updatenode c890f11ec01 configiba.1port
or
updatenode c890f11ec01 configiba.2ports


As a sample postscript, the netmask is set to default value: 255.255.255.0 and gateway is set to "X.X.255.254".  If the IB interface name is not a simple combination of short hostname and ibX or netmask and gateway cannot meet user's requirement, user can modify this scripts by himself, like the example below:

The node short hostname is 890f11ec01-en, and the IB interface name is 890f11ec01-ib0, c890f11ec01-ib1, etc. The user needs to modify 
my $hostname = "$ENV{NODE}-$nic";
to 
my $fullname = `echo $ENV{NODE} | cut -c 1-11`;
chomp($fullname);
my $hostname = "$fullname-$nic";

We assume every node have two IB adapters, if only one adapter is available on each node, please modify:
my @nums = (0..3);
to
my @nums = (0..1);

Note: Currently, for diskless installation, there is no /etc/resolv.conf  generated for the compute node.  This makes the compute node cannot resolve the name services on management node.  User needs to use rcp to copy resolv.conf to compute node as a workarround.  The fomat of resolv.conf is like:
domain ppd.pok.ibm.com
nameserver 172.16.0.1

Note: 172.16.0.1 is the IP address which provides the name service for IB interfaces.