mirror of
https://github.com/xcat2/xcat-core.git
synced 2025-07-11 15:21:29 +00:00
Merge pull request #147 from zet809/master
p8le hardware discovery document
This commit is contained in:
Xiaopeng Wang
@ -1,2 +0,0 @@
|
||||
Manually Define Nodes
|
||||
=====================
|
||||
@ -1,2 +0,0 @@
|
||||
MTMS-based Discovery
|
||||
====================
|
||||
@ -0,0 +1,80 @@
|
||||
Configure xCAT
|
||||
--------------
|
||||
|
||||
configure network table
|
||||
```````````````````````
|
||||
|
||||
|
||||
Normally, there will be at least two entries for the two subnet on MN in ``networks`` table after xCAT is installed::
|
||||
|
||||
#tabdump networks
|
||||
#netname,net,mask,mgtifname,gateway,dhcpserver,tftpserver,nameservers,ntpservers,logservers,dynamicrange,staticrange,staticrangeincrement,nodehostname,ddnsdomain,vlanid,domain,comments,disable
|
||||
"10_0_0_0-255_255_0_0","10.0.0.0","255.255.0.0","eth1","<xcatmaster>",,"10.0.1.1",,,,,,,,,,,,
|
||||
"50_0_0_0-255_255_0_0","50.0.0.0","255.255.0.0","eth2","<xcatmaster>",,"50.0.1.1",,,,,,,,,,,,
|
||||
|
||||
Pls run the following command to add networks in ``networks`` table if no entry in ``networks`` table::
|
||||
|
||||
makenetworks
|
||||
|
||||
setup DHCP
|
||||
``````````
|
||||
|
||||
Set the correct NIC from which DHCP server provide service::
|
||||
|
||||
chdef -t site dhcpinterfaces=eth1,eth2
|
||||
|
||||
Add dynamic range in purpose of assigning temporary IP adddress for FSP/BMCs and hosts::
|
||||
|
||||
chdef -t network 10_0_0_0-255_255_0_0 dynamicrange="10.0.100.1-10.0.100.100"
|
||||
chdef -t network 50_0_0_0-255_255_0_0 dynamicrange="50.0.100.1-50.0.100.100"
|
||||
|
||||
Update DHCP configuration file::
|
||||
|
||||
makedhcp -n
|
||||
makedhcp -a
|
||||
|
||||
setup DNS
|
||||
`````````
|
||||
|
||||
Set site.forwarders to your site-wide DNS servers that can resolve site or public hostnames. The DNS on the MN will forward any requests it can't answer to these servers::
|
||||
|
||||
chdef -t site forwarders=8.8.8.8
|
||||
|
||||
Run makedns to get the hostname/IP pairs copied from /etc/hosts to the DNS on the MN::
|
||||
|
||||
makedns -n
|
||||
|
||||
Config passwd table
|
||||
```````````````````
|
||||
|
||||
Set required passwords for xCAT to do hardware management and/or OS provisioning by adding entries to the xCAT ``passwd`` table::
|
||||
|
||||
# tabedit passwd
|
||||
# key,username,password,cryptmethod,authdomain,comments,disable
|
||||
|
||||
For hardware management with ipmi, add the following line::
|
||||
|
||||
"ipmi","ADMIN","admin",,,,
|
||||
|
||||
For OS provisioning, add the following line::
|
||||
|
||||
"system","root","cluster",,,,
|
||||
|
||||
Verify the genesis pkg
|
||||
``````````````````````
|
||||
|
||||
Genesis pkg is used to **create the root image for network boot** and it **MUST** be installed before doing hardware discovery.
|
||||
|
||||
* **[RH]**::
|
||||
|
||||
# rpm -qa |grep -i genesis
|
||||
xCAT-genesis-scripts-ppc64-2.10-snap201507240527.noarch
|
||||
xCAT-genesis-base-ppc64-2.10-snap201505172314.noarch
|
||||
|
||||
* **[ubuntu]**::
|
||||
|
||||
# dpkg -l | grep genesis
|
||||
ii xcat-genesis-base-ppc64 2.10-snap201505172314 all xCAT Genesis netboot image
|
||||
ii xcat-genesis-scripts 2.10-snap201507240105 ppc64el xCAT genesis
|
||||
|
||||
**Note:** If the two pkgs are not installed, pls installed them first and then run ``mknb ppc64`` to create the network boot root image.
|
||||
@ -1,6 +1,8 @@
|
||||
Hardware Discovery & Define Node
|
||||
================================
|
||||
|
||||
Hardware discovery is used to configure the FSP/BMC to get the hardware configuration information for the physical servers. The physical servers can be defined into xCAT database manually, or though our hardware discovery process. The available options for hardware discovery are MTMS based, switch based and sequential based.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
|
||||
@ -1 +1,37 @@
|
||||
.. include:: ../../common/discover/manually_define.rst
|
||||
Manually Define Nodes
|
||||
=====================
|
||||
|
||||
Manually define node means the admin know detailed information of the physical server and defines it into xCAT database with commands.
|
||||
|
||||
.. include:: schedule_environment.rst
|
||||
|
||||
Manually define node
|
||||
--------------------
|
||||
|
||||
To add a node object::
|
||||
|
||||
nodeadd cn1 groups=powerLE,all
|
||||
|
||||
Use the ``chdef`` command to add and change node attributes: ::
|
||||
|
||||
chdef cn1 mgt=ipmi cons=ipmi ip=10.0.101.1 netboot=petitboot
|
||||
chdef cn1 bmc=50.0.101.1 bmcusername=ADMIN bmcpassword=admin
|
||||
chdef cn1 installnic=mac primarynic=mac mac=6c:ae:8b:6a:d4:e4
|
||||
|
||||
The manually defined node will be like this::
|
||||
|
||||
# lsdef cn1
|
||||
Object name: cn1
|
||||
bmc=50.0.101.1
|
||||
bmcpassword=admin
|
||||
bmcusername=ADMIN
|
||||
cons=ipmi
|
||||
groups=powerLE,all
|
||||
installnic=mac
|
||||
ip=10.0.101.1
|
||||
mac=6c:ae:8b:6a:d4:e4
|
||||
mgt=ipmi
|
||||
netboot=petitboot
|
||||
postbootscripts=otherpkgs
|
||||
postscripts=syslog,remoteshell,syncfiles
|
||||
primarynic=mac
|
||||
|
||||
@ -1 +1,27 @@
|
||||
.. include:: ../../common/discover/manually_discovery.rst
|
||||
|
||||
If you have a few nodes which were not discovered by automated hardware discovery process, you could find them in ``discoverydata`` table using the nodediscoverls. The undiscovered nodes are those that have a discovery method value of 'undef' in the ``discoverydata`` table.
|
||||
|
||||
Display the undefined nodes with the ``nodediscoverls`` command::
|
||||
|
||||
#nodediscoverls -t undef
|
||||
UUID NODE METHOD MTM SERIAL
|
||||
fa2cec8a-b724-4840-82c7-3313811788cd undef undef 8247-22L 10112CA
|
||||
|
||||
If you want to manually define an 'undefined' node to a specific free node name, use the nodediscoverdef(TODO) command.
|
||||
|
||||
Before doing that, a node with desired IP address for host and FSP/BMC must be defined first::
|
||||
|
||||
nodeadd cn1 groups=powerLE,all
|
||||
chdef cn1 mgt=ipmi cons=ipmi ip=10.0.101.1 bmc=50.0.101.1 netboot=petitboot installnic=mac primarynic=mac
|
||||
|
||||
For example, if you want to assign the undefined node whose uuid is ``fa2cec8a-b724-4840-82c7-3313811788cd`` to cn1, run::
|
||||
|
||||
nodediscoverdef -u fa2cec8a-b724-4840-82c7-3313811788cd -n cn1
|
||||
|
||||
After manually defining it, the 'node name' and 'discovery method' attributes of the node will be changed. You can display the changed attributes using the ``nodediscoverls`` command::
|
||||
|
||||
#nodediscoverls
|
||||
UUID NODE METHOD MTM SERIAL
|
||||
fa2cec8a-b724-4840-82c7-3313811788cd cn1 manual 8247-22L 10112CA
|
||||
|
||||
|
||||
@ -1 +1,73 @@
|
||||
.. include:: ../../common/discover/mtms_discovery.rst
|
||||
MTMS-based Discovery
|
||||
====================
|
||||
|
||||
MTMS is short for Machine Type/Model and Serial which is unique for a physical server. The idea of MTMS based hardware discovery is that the admin know the physical location information of the server with specified MTMS. Then the admin can assign nodename and host ip address for the physical server.
|
||||
|
||||
.. include:: schedule_environment.rst
|
||||
.. include:: config_environment.rst
|
||||
|
||||
Discover server and define
|
||||
--------------------------
|
||||
|
||||
After environment is ready, and the server is powered, we can start server discovery process. The first thing to do is discovering the FSP/BMC of the server. It is automatically powered on when the physical server is powered.
|
||||
|
||||
The following command can be used to discovery FSP/BMC within an IP range and write the discovered node definition into a stanza file::
|
||||
|
||||
lsslp -s PBMC -u --range 50.0.100.1-100 -z > ./pbmc.stanza
|
||||
|
||||
You need to modify the node definition in stanza file before using them, the stanza file will be like this::
|
||||
|
||||
# cat pbmc.stanza
|
||||
cn1:
|
||||
objtype=node
|
||||
bmc=50.0.100.1
|
||||
nodetype=mp
|
||||
mtm=8247-42L
|
||||
serial=10112CA
|
||||
groups=pbmc,all
|
||||
mgt=ipmi
|
||||
hidden=0
|
||||
|
||||
Then, define it into xCATdb::
|
||||
|
||||
# cat pbmc.stanza | mkdef -z
|
||||
1 object definitions have been created or modified.
|
||||
|
||||
The server definition will be like this::
|
||||
|
||||
# lsdef cn1
|
||||
Object name: cn1
|
||||
bmc=50.0.100.1
|
||||
groups=pbmc,all
|
||||
hidden=0
|
||||
mgt=ipmi
|
||||
mtm=8247-42L
|
||||
nodetype=mp
|
||||
postbootscripts=otherpkgs
|
||||
postscripts=syslog,remoteshell,syncfiles
|
||||
serial=10112CA
|
||||
|
||||
After the physical server is defined into xCATdb, the next thing is update the node definition with the scheduled node info like this::
|
||||
|
||||
# chdef cn1 ip=10.0.101.1
|
||||
1 object definitions have been created or modified.
|
||||
|
||||
Then, add node info into /etc/hosts and DNS::
|
||||
|
||||
makehosts cn1
|
||||
makedns -n
|
||||
|
||||
Start discovery process
|
||||
-----------------------
|
||||
|
||||
To start discovery process, just need to power on the host remotely with the following command, and the discovery process will start automatically after the host is powered on::
|
||||
|
||||
rpower cn1 on
|
||||
|
||||
**[Optional]** If you'd like to monitor the discovery process, you can use::
|
||||
|
||||
chdef cn1 cons=ipmi
|
||||
makeconsercf
|
||||
rcons cn1
|
||||
|
||||
.. include:: standard_cn_definition.rst
|
||||
|
||||
@ -0,0 +1,38 @@
|
||||
Discover server and define
|
||||
--------------------------
|
||||
|
||||
After environment is ready, and the server is powered, we can start server discovery process. The first thing to do is discovering the FSP/BMC of the server. It is automatically powered on when the physical server is powered.
|
||||
|
||||
The following command can be used to discovery FSP/BMC within an IP range and write the discovered node definition into xCAT database::
|
||||
|
||||
lsslp -s PBMC -u --range 50.0.100.1-100 -w
|
||||
|
||||
The discovered PBMC node will be like this::
|
||||
|
||||
# lsdef Server-8247-22L-SN10112CA
|
||||
Object name: Server-8247-22L-SN10112CA
|
||||
bmc=50.0.100.1
|
||||
groups=pbmc,all
|
||||
hidden=0
|
||||
hwtype=pbmc
|
||||
mgt=ipmi
|
||||
mtm=8247-22L
|
||||
nodetype=mp
|
||||
postbootscripts=otherpkgs
|
||||
postscripts=syslog,remoteshell,syncfiles
|
||||
serial=10112CA
|
||||
|
||||
**Note**: Pls note that the PBMC node is just used to control the physical during hardware discovery process, it will be deleted after the correct server node object is found.
|
||||
|
||||
Start discovery process
|
||||
-----------------------
|
||||
|
||||
To start discovery process, just need to power on the PBMC node remotely with the following command, and the discovery process will start automatically after the host is powered on::
|
||||
|
||||
rpower Server-8247-22L-SN10112CA on
|
||||
|
||||
**[Optional]** If you'd like to monitor the discovery process, you can use::
|
||||
|
||||
chdef Server-8247-22L-SN10112CA cons=ipmi
|
||||
makeconsercf
|
||||
rcons Server-8247-22L-SN10112CA
|
||||
@ -0,0 +1,22 @@
|
||||
|
||||
In this document, the following configuration is used in the example
|
||||
|
||||
Management Node info::
|
||||
|
||||
MN Hostname: xcat1
|
||||
MN NIC info for Management Network(Host network): eth1, 10.0.1.1/16
|
||||
MN NIC info for Service Network(FSP/BMC nework): eth2, 50.0.1.1/16
|
||||
Dynamic IP range for Hosts: 10.0.100.1-10.0.100.100
|
||||
Dynamic IP range for FSP/BMC: 50.0.100.1-50.0.100.100
|
||||
|
||||
Compute Node info::
|
||||
|
||||
CN Hostname: cn1
|
||||
Machine type/model: 8247-22L
|
||||
Serial: 10112CA
|
||||
IP Address: 10.0.101.1
|
||||
Root Password: cluster
|
||||
Desired FSP/BMC IP Address: 50.0.101.1
|
||||
DHCP assigned FSP/BMC IP Address: 50.0.100.1
|
||||
FSP/BMC username: ADMIN
|
||||
FSP/BMC Password: admin
|
||||
@ -1 +1,51 @@
|
||||
.. include:: ../../common/discover/seq_discovery.rst
|
||||
|
||||
When the physical location of the server is not so important, sequential base hardware discovery can be used to simplify the discovery work. The idea is: providing a node pool, each node in the pool will be assigned an IP address for host and an IP address for FSP/BMC, then match first came physical server discovery request to the first free node in the node pool and configure the assigned IP address for host and FSP/BMC onto that pysical server.
|
||||
|
||||
.. include:: schedule_environment.rst
|
||||
.. include:: config_environment.rst
|
||||
|
||||
Prepare node pool
|
||||
-----------------
|
||||
|
||||
To prepare the node pool, shall predefine nodes first, then initialize the discovery process with the predefined nodes.
|
||||
|
||||
Predefine nodes
|
||||
```````````````
|
||||
|
||||
Predefine a group of node with desired IP address for host and IP address for FSP/BMC::
|
||||
|
||||
nodeadd cn1 groups=powerLE,all
|
||||
chdef cn1 mgt=ipmi cons=ipmi ip=10.0.101.1 bmc=50.0.101.1 netboot=petitboot installnic=mac primarynic=mac
|
||||
|
||||
Initialize the discovery process
|
||||
````````````````````````````````
|
||||
|
||||
Specify the predefined nodes to the nodediscoverstart command to initialize the discovery process::
|
||||
|
||||
nodediscoverstart noderange=cn1
|
||||
|
||||
Pls see "nodediscoverstart man page<TBD>" for more details.
|
||||
|
||||
Display information about the discovery process
|
||||
```````````````````````````````````````````````
|
||||
|
||||
There are additional nodediscover commands you can run during the discovery process. See their man pages for more details.
|
||||
|
||||
|
||||
Verify the status of discovery::
|
||||
|
||||
nodediscoverstatus
|
||||
|
||||
Show the nodes that have been discovered so far::
|
||||
|
||||
nodediscoverls -t seq -l
|
||||
|
||||
Stop the current sequential discovery process::
|
||||
|
||||
nodediscoverstop
|
||||
|
||||
Note: The sequential discovery process will be stopped automatically when all of the node names in the node pool are used up.
|
||||
|
||||
.. include:: pbmc_discovery.rst
|
||||
.. include:: standard_cn_definition.rst
|
||||
|
||||
@ -0,0 +1,25 @@
|
||||
Verify node definition
|
||||
----------------------
|
||||
|
||||
The following is an example of the server node definition after hardware discovery::
|
||||
|
||||
# lsdef cn1
|
||||
Object name: cn1
|
||||
arch=ppc64
|
||||
bmc=50.0.100.1
|
||||
cons=ipmi
|
||||
cpucount=192
|
||||
cputype=POWER8E (raw), altivec supported
|
||||
groups=powerLE,all
|
||||
installnic=mac
|
||||
ip=10.0.101.1
|
||||
mac=6c:ae:8b:02:12:50
|
||||
memory=65118MB
|
||||
mgt=ipmi
|
||||
mtm=8247-22L
|
||||
netboot=petitboot
|
||||
postbootscripts=otherpkgs
|
||||
postscripts=syslog,remoteshell,syncfiles
|
||||
primarynic=mac
|
||||
serial=10112CA
|
||||
supportedarchs=ppc64
|
||||
@ -1 +1,89 @@
|
||||
.. include:: ../../common/discover/switch_discovery.rst
|
||||
|
||||
For switch based hardware discovery, the server are identified though the switches and switchposts they directly connect to.
|
||||
|
||||
.. include:: schedule_environment.rst
|
||||
|
||||
Switch info::
|
||||
|
||||
Switch name: switch1
|
||||
Switch username: xcat
|
||||
Switch password: passw0rd
|
||||
Switch IP Address: 10.0.201.1
|
||||
Switch port for Compute Node: port0
|
||||
|
||||
.. include:: config_environment.rst
|
||||
|
||||
Predefined Nodes
|
||||
---------------
|
||||
|
||||
In order to differentiate one node from another, the admin needs to predefine node in xCAT database based on the switches information. This consists of two parts:
|
||||
|
||||
#. :ref:`predefined_switches_label`
|
||||
#. :ref:`predefined_server_nodes_label`
|
||||
|
||||
|
||||
.. _predefined_switches_label:
|
||||
|
||||
Predefine Switches
|
||||
|
||||
The predefined switches will represent devices that the physical servers are connected to. xCAT need to access those switches to get server related information through SNMP v3.
|
||||
|
||||
So the admin need to make sure those switches are configured correctly with SNMP v3 enabled. <TODO: The document that Configure Ethernet Switches>
|
||||
|
||||
Then, define switch info into xCAT::
|
||||
|
||||
nodeadd switch1 groups=switch,all
|
||||
chdef switch1 ip=10.0.201.1
|
||||
tabch switch=switch1 switches.snmpversion=3 switches.username=xcat switches.password=passw0rd switches.auth=sha
|
||||
|
||||
Add switch into DNS using the following commands::
|
||||
|
||||
makehosts switch1
|
||||
makedns -n
|
||||
|
||||
.. _predefined_server_nodes_label:
|
||||
|
||||
Predefine Server node
|
||||
|
||||
After switches are defined, the server node can be predefined with the following commands::
|
||||
|
||||
nodeadd cn1 groups=powerLE,all
|
||||
chdef cn1 mgt=ipmi cons=ipmi ip=10.0.101.1 bmc=50.0.101.1 netboot=petitboot installnic=mac primarynic=mac
|
||||
chdef cn1 switch=switch1 switchport=0
|
||||
|
||||
Add cn1 into DNS::
|
||||
|
||||
makehosts cn1
|
||||
maekdns -n
|
||||
|
||||
.. include:: pbmc_discovery.rst
|
||||
|
||||
Verify node definition
|
||||
----------------------
|
||||
|
||||
The following is an example of the server node definition after hardware discovery::
|
||||
|
||||
|
||||
#lsdef cn1
|
||||
Object name: cn1
|
||||
arch=ppc64
|
||||
bmc=50.0.101.1
|
||||
cons=ipmi
|
||||
cpucount=192
|
||||
cputype=POWER8E (raw), altivec supported
|
||||
groups=powerLE,all
|
||||
installnic=mac
|
||||
ip=10.0.101.1
|
||||
mac=6c:ae:8b:02:12:50
|
||||
memory=65118MB
|
||||
mgt=ipmi
|
||||
mtm=8247-22L
|
||||
netboot=petitboot
|
||||
postbootscripts=otherpkgs
|
||||
postscripts=syslog,remoteshell,syncfiles
|
||||
primarynic=mac
|
||||
serial=10112CA
|
||||
supportedarchs=ppc64
|
||||
switch=switch1
|
||||
switchport=0
|
||||
|
||||
Reference in New Issue
Block a user