mirror of
https://github.com/xcat2/xcat-core.git
synced 2025-06-12 00:00:12 +00:00
add doc for xCAT basic concepts
This commit is contained in:
immarvin
GONG Jie
@ -1,4 +1,134 @@
|
||||
Global Configuration
|
||||
====================
|
||||
|
||||
Configure the **master**, **domain** ...
|
||||
All the xCAT global configurations are stored in site table, xCAT Admin can adjust the configuration by modifing the site attibute with ``tabedit``.
|
||||
|
||||
This section only presents some key global configurations, for the complete reference on the xCAT global configurations, please refer to the ``tabdump -d site``.
|
||||
|
||||
|
||||
DATABASE ATTRIBUTES
|
||||
-------------------
|
||||
|
||||
* excludenodes:
|
||||
A set of comma separated nodes and/or groups that would automatically be subtracted from any noderange, it can be used for excluding some failed nodes for any xCAT commands. See the [TODO noderange] manpage for details on supported formats.
|
||||
|
||||
* nodestatus:
|
||||
If set to ``n``, the ``nodelist.status`` column will not be updated during the node deployment, node discovery and power operations. The default is to update.
|
||||
|
||||
|
||||
DHCP ATTRIBUTES
|
||||
---------------
|
||||
|
||||
* dhcpinterfaces:
|
||||
The network interfaces DHCP should listen on. If it is the same for all nodes, use a simple comma-separated list of NICs. To specify different NICs for different nodes ::
|
||||
|
||||
xcatmn|eth1,eth2;service|bond0.
|
||||
|
||||
In this example xcatmn is the name of the xCAT MN, and DHCP there should listen on eth1 and eth2. On all of the nodes in group ``service`` DHCP should listen on the bond0 nic.
|
||||
|
||||
* dhcplease:
|
||||
The lease time for the dhcp client. The default value is 43200.
|
||||
|
||||
|
||||
* managedaddressmode:
|
||||
The mode of networking configuration during node provision.
|
||||
If set to ``static``, the network configuration will be configured in static mode based on the node and network definition on MN.
|
||||
If set to ``dhcp``, the network will be configured with dhcp protocol.
|
||||
The default is ``dhcp``.
|
||||
|
||||
|
||||
DNS ATTRIBUTES
|
||||
--------------
|
||||
|
||||
* domain:
|
||||
The DNS domain name used for the cluster.
|
||||
|
||||
* forwarders:
|
||||
The DNS servers at your site that can provide names outside of the cluster. The ``makedns`` command will configure the DNS on the management node to forward requests it does not know to these servers.
|
||||
Note that the DNS servers on the service nodes will ignore this value and always be configured to forward requests to the management node.
|
||||
|
||||
* master:
|
||||
The hostname of the xCAT management node, as known by the nodes.
|
||||
|
||||
* nameservers:
|
||||
A comma delimited list of DNS servers that each node in the cluster should use. This value will end up in the nameserver settings of the ``/etc/resolv.conf`` on each node. It is common (but not required) to set this attribute value to the IP addr of the xCAT management node, if you have set up the DNS on the management node by running ``makedns``. In a hierarchical cluster, you can also set this attribute to ``<xcatmaster>`` to mean the DNS server for each node should be the node that is managing it (either its service node or the management
|
||||
node).
|
||||
|
||||
|
||||
* dnsinterfaces:
|
||||
The network interfaces DNS server should listen on. If it is the same for all nodes, use a simple comma-separated list of NICs. To specify different NICs for different nodes ::
|
||||
|
||||
xcatmn|eth1,eth2;service|bond0.
|
||||
|
||||
In this example xcatmn is the name of the xCAT MN, and DNS there should listen on eth1 and eth2. On all of the nodes in group ``service`` DNS should listen on the bond0 nic.
|
||||
**NOTE**: if using this attribute to block certain interfaces, make sure the ip maps to your hostname of xCAT MN is not blocked since xCAT needs to use this ip to communicate with the local NDS server on MN.
|
||||
|
||||
|
||||
INSTALL/DEPLOYMENT ATTRIBUTES
|
||||
-----------------------------
|
||||
|
||||
* installdir:
|
||||
The local directory name used to hold the node deployment packages.
|
||||
|
||||
* runbootscripts:
|
||||
If set to ``yes`` the scripts listed in the postbootscripts attribute in the osimage and postscripts tables will be run during each reboot of stateful (diskful) nodes. This attribute has no effect on stateless and statelite nodes. Please run the following command after you change the value of this attribute ::
|
||||
|
||||
updatenode <nodes> -P setuppostbootscripts
|
||||
|
||||
* precreatemypostscripts:
|
||||
(``yes/1`` or ``no/0``). Default is ``no``. If yes, it will instruct xCAT at ``nodeset`` and ``updatenode`` time to query the db once for all of the nodes passed into the cmd and create the mypostscript file for each node, and put them in a directory of tftpdir(such as: /tftpboot). If no, it will not generate the mypostscript file in the ``tftpdir``.
|
||||
|
||||
* xcatdebugmode:
|
||||
the value of xcat debug mode. Currently supported values: ``0: off(default); 1: on``
|
||||
|
||||
|
||||
REMOTESHELL ATTRIBUTES
|
||||
----------------------
|
||||
|
||||
* sshbetweennodes:
|
||||
Comma separated list of groups of compute nodes to enable passwordless root ssh during install, or ``xdsh -K``. Default is ``ALLGROUPS``. Set to ``NOGROUPS``,if you do not wish to enabled any group of compute nodes.If using the ``zone`` table, this attribute in not used.
|
||||
|
||||
|
||||
SERVICES ATTRIBUTES
|
||||
-------------------
|
||||
|
||||
* consoleondemand:
|
||||
When set to ``yes``, conserver connects and creates the console output only when the user opens the console. Default is ``no`` on Linux, yes on AIX.
|
||||
|
||||
* timezone:
|
||||
the timezone for all the nodes in the cluster(e.g. ``America/New_York``).
|
||||
|
||||
* tftpdir:
|
||||
tftp directory path. Default is /tftpboot.
|
||||
|
||||
* tftpflags:
|
||||
The flags that used to start tftpd. Default is ``-v -l -s /tftpboot -m /etc/tftpmapfile4xcat.conf`` if ``tftplfags`` is not set.
|
||||
|
||||
|
||||
VIRTUALIZATION ATTRIBUTES
|
||||
--------------------------
|
||||
|
||||
* persistkvmguests:
|
||||
Keep the kvm definition on the kvm hypervisor when you power off the kvm guest node. This is useful for you to manually change the kvm xml definition file in ``virsh`` for debugging. Set anything means ``enable``.
|
||||
|
||||
|
||||
XCAT DAEMON ATTRIBUTES
|
||||
----------------------
|
||||
|
||||
* xcatdport:
|
||||
The port used by the xcatd daemon for client/server communication.
|
||||
|
||||
* xcatiport:
|
||||
The port used by xcatd to receive install status updates from nodes.
|
||||
|
||||
* xcatlport:
|
||||
The port used by xcatd command log writer process to collect command output.
|
||||
|
||||
* xcatsslversion:
|
||||
The ssl version by xcatd. Default is ``SSLv3``.
|
||||
|
||||
* xcatsslciphers:
|
||||
The ssl cipher by xcatd. Default is ``3DES``.
|
||||
|
||||
|
||||
|
||||
|
||||
@ -1,6 +1,10 @@
|
||||
Basic Concepts
|
||||
==============
|
||||
|
||||
Most xCAT data, including global configuration and cluster information, are stored in xCAT tables in the databse. xCAT abstracts serveral object types from the cluster information to perform the cluster management work.
|
||||
|
||||
This section presents some basic xCAT knowledge, including xCAT global configuration, the description of xCAT object types and database tables, some commands and techniques on xCAT objects and databse tables, the network services used in xCAT and the typical network planning for a xCAT managed cluster.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
|
||||
@ -1,8 +1,12 @@
|
||||
Network Planning
|
||||
================
|
||||
|
||||
This section introduces xCAT node types, the network services in xCAT, the network structure and present some considerations on planning the xCAT network.
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
node_type.rst
|
||||
network_service/index.rst
|
||||
network_cfg/index.rst
|
||||
|
||||
@ -1,2 +1,12 @@
|
||||
Network Configuration
|
||||
=====================
|
||||
|
||||
This section introduces the network structure in xCAT and present some consideration on planning the xCAT network.
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
xcat_network.rst
|
||||
xcat_net_planning.rst
|
||||
|
||||
|
||||
@ -0,0 +1,63 @@
|
||||
xCAT Network Planning
|
||||
=====================
|
||||
|
||||
Before setting up your cluster, there are a few things that are important to think through first, because it is much easier to go in the direction you want right from the beginning, instead of changing course midway through.
|
||||
|
||||
Do You Need Hierarchy in Your Cluster?
|
||||
--------------------------------------
|
||||
|
||||
Service Nodes
|
||||
`````````````
|
||||
For very large clusters, xCAT has the ability to distribute the management operations to service nodes. This allows the management node to delegate all management responsibilities for a set of compute or storage nodes to a service node so that the management node doesn't get overloaded. Although xCAT automates a lot of the aspects of deploying and configuring the services, it still adds complexity to your cluster. So the question is: at what size cluster do you need to start using service nodes?? The exact answer depends on a lot of factors (mgmt node size, network speed, node type, OS, frequency of node deployment, etc.), but here are some general guidelines for how many nodes a single mgmt node (or single service node) can handle:
|
||||
|
||||
Linux:
|
||||
Stateful or Stateless: 500 nodes
|
||||
Statelite: 250 nodes
|
||||
AIX:
|
||||
150 nodes
|
||||
|
||||
These numbers can be higher (approximately double) if you are willing to "stage" the more intensive operations, like node deployment.
|
||||
|
||||
Of course, there are some reasons to use service nodes that are not related to scale, for example, if some of your nodes are far away (network-wise) from the mgmt node.
|
||||
|
||||
Network Hierarchy
|
||||
`````````````````
|
||||
For large clusters, you may want to divide the management network into separate subnets to limit the broadcast domains. (Service nodes and subnets don't have to coincide, although they often do.) xCAT clusters as large as 3500 nodes have used a single broadcast domain.
|
||||
|
||||
Some cluster administrators also choose to sub-divide the application interconnect to limit the network contention between separate parallel jobs.
|
||||
|
||||
|
||||
Design an xCAT Cluster for High Availability
|
||||
--------------------------------------------
|
||||
|
||||
Everyone wants their cluster to be as reliable and available as possible, but there are multiple ways to achieve that end goal. Availability and complexity are inversely proportional. You should choose an approach that balances these 2 in a way that fits your environment the best. Here's a few choices in order of least complex to more complex.
|
||||
|
||||
Service Node Pools With No HA Software
|
||||
``````````````````````````````````````
|
||||
Service node pools is an xCAT approach in which more than one service node (SN) is in the broadcast domain for a set of nodes. When each node netboots, it chooses an available SN by which one responds to its DHCP request 1st. When services are set up on the node (e.g. DNS), xCAT configures the services to use at that SN and one other SN in the pool. That way, if one SN goes down, the node can keep running, and the next time it netboots it will automatically choose another SN.
|
||||
|
||||
This approach is most often used with stateless nodes because that environment is more dynamic. It can possibly be used with stateful nodes (with a little more effort), but that type of node doesn't netboot nearly as often so a more manual operation (snmove) is needed in that case move a node to different SNs.
|
||||
|
||||
It is best to have the SNs be as robust as possible, for example, if they are diskfull, configure them with at least 2 disks that are RAID'ed together.
|
||||
|
||||
In smaller clusters, the management node (MN) can be part of the SN pool with one other SN.
|
||||
|
||||
In larger clusters, if the network topology dictates that the MN is only for managing the SNs (not the compute nodes), then you need a plan for what to do if the MN fails. Since the cluster can continue to run if the MN is down temporarily, the plan could be as simple as have a backup MN w/o any disks. If the primary MN fails, move its RAID'ed disks to the backup MN and power it on.
|
||||
|
||||
HA Management Node
|
||||
``````````````````
|
||||
|
||||
If you want to use HA software on your management node to synchronize data and fail over services to a backup MN, see [TODO Highly_Available_Management_Node], which discusses the different options and the pros and cons.
|
||||
|
||||
It is important to note that some HA-related software like DRDB, Pacemaker, and Corosync is not officially supported by IBM, meaning that if you have a problem specifically with that software, you will have to go to the open source community or another vendor to get a fix.
|
||||
|
||||
HA Service Nodes
|
||||
````````````````
|
||||
|
||||
When you have NFS-based diskless (statelite) nodes, there is sometimes the motivation make the NFS serving highly available among all of the service nodes. This is not recommended because it is a very complex configuration. In our opinion, the complexity of this setup can nullify much of the availibility you hope to gain. If you need your compute nodes to be highly available, you should strongly consider stateful or stateless nodes.
|
||||
|
||||
If you still have reasons to pursue HA service nodes:
|
||||
|
||||
* For AIX, see [TODO XCAT_HASN_with_GPFS]
|
||||
* For linux, a couple prototype clusters have been set up in which the NFS service on the SNs is provided by GPFS CNFS (Clustered NFS). A howto is being written to describe the setup as an example. Stay tuned.
|
||||
|
||||
@ -0,0 +1,22 @@
|
||||
Networks in an xCAT Cluster
|
||||
===========================
|
||||
|
||||
The networks that are typically used in a cluster are:
|
||||
|
||||
Management network
|
||||
------------------
|
||||
used by the management node to install and manage the OS of the nodes. The MN and in-band NIC of the nodes are connected to this network. If you have a large cluster with service nodes, sometimes this network is segregated into separate VLANs for each service node.
|
||||
|
||||
Service network
|
||||
---------------
|
||||
used by the management node to control the nodes out of band via the BMC. If the BMCs are configured in shared mode, then this network can be combined with the management network.
|
||||
|
||||
Application network
|
||||
-------------------
|
||||
used by the HPC applications on the compute nodes. Usually an IB network.
|
||||
|
||||
Site (Public) network
|
||||
---------------------
|
||||
used to access the management node and sometimes for the compute nodes to provide services to the site.
|
||||
|
||||
|
||||
Binary file not shown.
@ -1,4 +1,35 @@
|
||||
Network Services
|
||||
================
|
||||
|
||||
The following network services are used by xCAT...
|
||||
The following network services are used by xCAT:
|
||||
|
||||
DNS(Domain Name Service)
|
||||
------------------------
|
||||
The dns server, usually the management node or service node, provides the domain name service for the entire cluster.
|
||||
|
||||
HTTP(HyperText Transfer Protocol)
|
||||
---------------------------------
|
||||
The http server,usually the management node or service node, acts as the download server for the initrd and kernel, the configuration file for the installer and repository for the online installation.
|
||||
|
||||
DHCP(Dynamic Host Configuration Protocol)
|
||||
-----------------------------------------
|
||||
The dhcp server,usually the management node or service node, privides the dhcp service for the entire cluster.
|
||||
|
||||
TFTP(Trivial File Transfer Protocol)
|
||||
------------------------------------
|
||||
The tftp server,usually the management node or service node, acts as the download server for bootloader binaries, bootloader configuration file, initrd and kernel.
|
||||
|
||||
NFS(Network File System)
|
||||
------------------------
|
||||
The NFS server, usually the management node or service node, provides the file system sharing between the management node and service node, or persistent file system support for the stateless node.
|
||||
|
||||
NTP(Network Time Protocol)
|
||||
--------------------------
|
||||
The NTP server, usually the management node or service node, provide the network time service for the entire cluster.
|
||||
|
||||
SYSLOG
|
||||
------
|
||||
Usually, xCAT uses rsyslog as the syslog service for the cluster, all the log messages of the nodes in the cluster are forwarded to the management node.
|
||||
|
||||
|
||||
|
||||
|
||||
@ -0,0 +1,38 @@
|
||||
xCAT Cluster Node Types
|
||||
=======================
|
||||
|
||||
This section describes 2 standard node types xCAT supports, gives the pros and cons of each, and describes the cluster characteristics that will result from each.
|
||||
|
||||
Stateful (diskfull)
|
||||
-------------------
|
||||
|
||||
traditional cluster with OS on each node's local disk.
|
||||
|
||||
Main advantage
|
||||
``````````````
|
||||
this approach is familiar to most admins, and they typically have many years of experience with it
|
||||
|
||||
Main disadvantage
|
||||
`````````````````
|
||||
you have to manage all of the individual OS copies
|
||||
|
||||
|
||||
Stateless(diskless)
|
||||
-------------------
|
||||
|
||||
nodes boot from a RAMdisk OS image downloaded from the xCAT mgmt node or service node at boot time. (This option is not available on AIX).
|
||||
|
||||
Main advantage
|
||||
``````````````
|
||||
central management of OS image, but nodes are not tethered to the mgmt node or service node it booted from
|
||||
|
||||
Main disadvantage
|
||||
`````````````````
|
||||
you can't use a large image with many different applications all in the image for varied users, because it uses too much of the node's memory to store the ramdisk. (To mitigate this disadvantage, you can put your large application binaries and libraries in gpfs to reduce the ramdisk size. This requires some manual configuration of the image).
|
||||
|
||||
* Scratch disk:
|
||||
Each node can also have a local "scratch" disk for ``swap``, ``/tmp``, ``/var``, ``log`` files, dumps, etc. The purpose of the scratch disk is to provide a location for files that are written to by the node that can become quite large or for files that you don't want to have disappear when the node reboots. There should be nothing put on the scratch disk that represents the node's "state", so that if the disk fails you can simply replace it and reboot the node. A scratch disk would typically be used for situations like: job scheduling preemption is required (which needs a lot of swap space), the applications write large temp files, or you want to keep gpfs log or trace files persistently. (As a partial alternative to using the scratch disk, customers can choose to put ``/tmp`` ``/var/tmp``, and log files (except GPFS logs files) in GPFS, but must be willing to accept the dependency on GPFS).
|
||||
|
||||
* Statelite persistent files:
|
||||
xCAT supports layering some statelite persistent files/dirs on top of a ramdisk node. The statelite persistent files are nfs mounted. In this case, as little as possible should be in statelite persistent files, at least nothing that will cause the node to hang if the nfs mount goes away.
|
||||
|
||||
@ -0,0 +1,17 @@
|
||||
Database Commands
|
||||
=================
|
||||
|
||||
There are 5 database related commands in xCAT:
|
||||
|
||||
* ``tabdump`` : Displays the header and all the rows of the specified table in CSV (comma separated values) format.
|
||||
|
||||
* ``tabedit`` : Opens the specified table in the user's editor, allows them to edit any text, and then writes changes back to the database table. The table is flattened into a CSV (comma separated values) format file before giving it to the editor. After the editor is exited, the CSV file will be translated back into the database format.
|
||||
|
||||
* ``tabgrep`` : List table names in which an entry for the given node appears.
|
||||
|
||||
* ``dumpxCATdb`` : Dumps all the xCAT db tables to CSV files under the specified directory, often used to backup the xCAT database in xCAT reinstallation or management node migration.
|
||||
|
||||
* ``restorexCATdb`` : Restore the xCAT db tables with the CSV files under the specified directory.
|
||||
|
||||
|
||||
For the complete reference on all the xCAT database related commands, please refer to the xCAT manpage with ``man <command>``
|
||||
@ -0,0 +1,28 @@
|
||||
Key xCAT Tables
|
||||
===============
|
||||
|
||||
They are many tables in xCAT databse to store various categories of information. This section only introduces several key xCAT tables which need to be initialized or viewed explicitly. For the complete reference on xCAT tables, please refer to the page <todo> or run ``tabdump -d <table name>``.
|
||||
|
||||
site
|
||||
----
|
||||
Global settings for the whole cluster. This table is different from the other tables in that each attribute is just named in the key column, rather than having a separate column for each attribute. Refer to the :doc:`Global Configuration </guides/admin-guides/basic_concepts/global_cfg/index>` page for the global attributes.
|
||||
|
||||
policy
|
||||
------
|
||||
Controls who has authority to run specific xCAT operations. It is basically the Access Control List (ACL) for xCAT. It is sorted on the priority field before evaluating. Please run ``tabdump -d policy`` for details.
|
||||
|
||||
passwd
|
||||
------
|
||||
Contains default userids and passwords for xCAT to access cluster components. In most cases, xCAT will also actually set the userid/password in the relevant component when it is being configured or installed. Userids/passwords for specific cluster components can be overidden in other tables, e.g. ``mpa`` , ``ipmi`` , ``ppchcp`` , etc.
|
||||
|
||||
networks
|
||||
--------
|
||||
Describes the networks in the cluster and info necessary to set up nodes on that network.
|
||||
|
||||
auditlog
|
||||
--------
|
||||
Contains the audit log data.
|
||||
|
||||
eventlog
|
||||
--------
|
||||
Stores the events occurred.
|
||||
@ -1,20 +1,16 @@
|
||||
Database
|
||||
========
|
||||
|
||||
xCAT stores all the persistent data used to manage clusters in a database.
|
||||
xCAT stores all the persistent data including global configuration, user settings and cluster information in a database.
|
||||
|
||||
As the administrator, you can use a set of table oriented commands to modify the data (**recommended**), or directly modify the table using the ``tabedit`` command.
|
||||
This section introduces some database related xCAT commands, some key xCAT database tables and, the usage of regular expressions in xCAT database.
|
||||
|
||||
For a more indepth overview see the manpage for xcatdb: ``man xcatdb``
|
||||
For a complete reference, see the manpage for xcatdb: ``man xcatdb``
|
||||
|
||||
Commands
|
||||
--------
|
||||
|
||||
Object Definitions
|
||||
------------------
|
||||
|
||||
Node Groups
|
||||
-----------
|
||||
|
||||
Regular Expressions
|
||||
-------------------
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
dbcmd.rst
|
||||
dbtables.rst
|
||||
regexp_db.rst
|
||||
|
||||
@ -0,0 +1,86 @@
|
||||
GROUPS AND REGULAR EXPRESSIONS IN TABLES
|
||||
========================================
|
||||
|
||||
Using Regular Expressions in the xCAT Tables
|
||||
--------------------------------------------
|
||||
|
||||
The xCAT database has a number of tables, some with rows that are keyed by node name (such as ``noderes`` and ``nodehm`` ) and others that are not keyed by node name (for example, the ``policy`` table). The tables that are keyed by node name have some extra features that enable a more template-based style to be used:
|
||||
|
||||
Any group name can be used in lieu of a node name in the node field, and that row will then provide "default" attribute values for any node in that group. A row with a specific node name can then override one or more attribute values for that specific node. For example, if the ``nodehm`` table contains ::
|
||||
|
||||
#node,power,mgt,cons,termserver,termport,conserver,serialport,serialspeed,serialflow,getmac,cmdmapping,comments,disable
|
||||
"mygroup",,"ipmi",,,,,,"19200",,,,,
|
||||
"node1",,,,,,,,"115200",,,,,
|
||||
|
||||
In the above example, the node group called "mygroup" sets ``mgt=ipmi`` and ``serialspeed=19200``. Any nodes that are in this group will have those attribute values, unless overridden. For example, if "node2" is a member of "mygroup", it will automatically inherit these attribute values (even though it is not explicitly listed in this table). In the case of "node1" above, it inherits ``mgt=ipmi``, but overrides the ``serialspeed`` to be 115200, instead of 19200. A useful, typical way to use this capability is to create a node group for your nodes and for all the attribute values that are the same for every node, set them at the group level. Then you only have to set attributes for each node that vary from node to node.
|
||||
|
||||
xCAT extends the group capability so that it can also be used for attribute values that vary from node to node in a very regular pattern. For example, if in the ``ipmi`` table you want the ``bmc`` attribute to be set to whatever the nodename is with "-bmc" appended to the end of it, then use this in the ``ipmi`` table ::
|
||||
|
||||
#node,bmc,bmcport,taggedvlan,bmcid,username,password,comments,disable
|
||||
"compute","/\z/-bmc/",,,,,,,
|
||||
|
||||
In this example, "compute" is a node group that contains all of the compute nodes. The 2nd attribute (``bmc``) is a regular expression that is similar to a substitution pattern. The 1st part ``\z`` matches the end of the node name and substitutes ``-bmc``, effectively appending it to the node name.
|
||||
|
||||
Another example is if "node1" is assigned the IP address "10.0.0.1", node2 is assigned the IP address "10.0.0.2", etc., then this could be represented in the ``hosts`` table with the single row ::
|
||||
|
||||
#node,ip,hostnames,otherinterfaces,comments,disable
|
||||
"compute","|node(\d+)|10.0.0.($1+0)|",,,,
|
||||
|
||||
In this example, the regular expression in the ``ip`` attribute uses ``|`` to separate the 1st and 2nd part. This means that xCAT will allow arithmetic operations in the 2nd part. In the 1st part, ``(\d+)``, will match the number part of the node name and put that in a variable called ``$1``. The 2nd part is what value to give the ``ip`` attribute. In this case it will set it to the string "10.0.0." and the number that is in ``$1``. (Zero is added to ``$1`` just to remove any leading zeroes.)
|
||||
|
||||
A more involved example is with the ``mp`` table. If your blades have node names node01, node02, etc., and your chassis node names are cmm01, cmm02, etc., then you might have an ``mp`` table like ::
|
||||
|
||||
#node,mpa,id,nodetype,comments,disable
|
||||
"blade","|\D+(\d+)|cmm(sprintf('%02d',($1-1)/14+1))|","|\D+(\d+)|(($1-1)%14+1)|",,
|
||||
|
||||
Before you panic, let me explain each column:
|
||||
|
||||
``blade``
|
||||
|
||||
This is a group name. In this example, we are assuming that all of your blades belong to this group. Each time the xCAT software accesses the ``mp`` table to get the management module and slot number of a specific blade (e.g. node20), this row will match (because node20 is in the blade group). Once this row is matched for node20, then the processing described in the following items will take place.
|
||||
|
||||
``|\D+(\d+)|cmm(sprintf('%02d',($1-1)/14+1))|``
|
||||
|
||||
This is a perl substitution pattern that will produce the value for the second column of the table (the management module hostname). The text ``\D+(\d+)`` between the 1st two vertical bars is a regular expression that matches the node name that was searched for in this table (in this example node20). The text that matches within the 1st set of parentheses is set to ``$1``. (If there was a 2nd set of parentheses, it would be set to ``$2``, and so on.) In our case, the ``\D+`` matches the non-numeric part of the name (node) and the ``\d+`` matches the numeric part (20). So ``$1`` is set to 20. The text ``cmm(sprintf('%02d',($1-1)/14+1))`` between the 2nd and 3rd vertical bars produces the string that should be used as the value for the ``mpa`` attribute for node20. Since ``$1`` is set to 20, the expression ``($1-1)/14+1`` equals "19/14 + 1", which equals "2". (The division is integer division, so "19/14" equals 1. Fourteen is used as the divisor, because there are 14 blades in each chassis.) The value of 2 is then passed into sprintf() with a format string to add a leading zero, if necessary, to always make the number two digits. Lastly the string cmm is added to the beginning, making the resulting string ``cmm02``, which will be used as the hostname of the management module.
|
||||
|
||||
``|\D+(\d+)|(($1-1)%14+1)|``
|
||||
|
||||
This item is similar to the one above. This substituion pattern will produce the value for the 3rd column (the chassis slot number for this blade). Because this row was the match for node20, the parentheses within the 1st set of vertical bars will set ``$1`` to 20. Since ``%`` means modulo division, the expression ``($1-1)%14+1`` will evaluate to 6.
|
||||
|
||||
See `perlre <http://www.perl.com/doc/manual/html/pod/perlre.html>`_ for information on perl regular expressions.
|
||||
|
||||
|
||||
Easy Regular expressions
|
||||
------------------------
|
||||
|
||||
As of xCAT 2.8.1, you can use a modified version of the regular expression support described in the previous section. You do not need to enter the node information (1st part of the expression), it will be derived from the input nodename. You only need to supply the 2nd part of the expression to determine the value to give the attribute.
|
||||
|
||||
For example:
|
||||
|
||||
If node1 is assigned the IP address 10.0.0.1, node2 is assigned the IP address 10.0.0.2, etc., then this could be represented in the ``hosts`` table with the single row:
|
||||
|
||||
Using full regular expression support you would put this in the ``hosts`` table. ::
|
||||
|
||||
chdef -t group compute ip="|node(\d+)|10.0.0.($1+0)|"
|
||||
tabdump hosts
|
||||
#node,ip,hostnames,otherinterfaces,comments,disable
|
||||
"compute","|node(\d+)|10.0.0.($1+0)|",,,,
|
||||
|
||||
Using easy regular expression support you would put this in the hosts table. ::
|
||||
|
||||
chdef -t group compute ip="|10.0.0.($1+0)|"
|
||||
tabdump hosts
|
||||
#node,ip,hostnames,otherinterfaces,comments,disable
|
||||
"compute","|10.0.0.($1+0)|",,,,
|
||||
|
||||
In the easy regx example, the expression only has the 2nd part of the expression from the previous example. xCAT will evaluate the node name, matching the number part of the node name, and create the 1st part of the expression . The 2nd part supplied is what value to give the ip attribute. The resulting output is the same.
|
||||
|
||||
|
||||
Verify your regular expression
|
||||
------------------------------
|
||||
|
||||
After you create your table with regular expression, make sure they are evaluating as you expect. ::
|
||||
|
||||
lsdef node1 | grep ip
|
||||
ip=10.0.0.1
|
||||
|
||||
Binary file not shown.
@ -0,0 +1,13 @@
|
||||
Basic xCAT object types
|
||||
=======================
|
||||
|
||||
This section introduces the description and key attributes of several basic xCAT object types. For the complete description of all the xCAT object types, please refer to the [TODO:guides/admin-guides/references/index.html#xcat-man-pages] or run command ``man xcatdb``, for details of a specified xCAT object type, please run command ``man <object type>``.
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
node.rst
|
||||
group.rst
|
||||
osimage.rst
|
||||
|
||||
@ -1,2 +1,68 @@
|
||||
``group``
|
||||
=========
|
||||
group
|
||||
=====
|
||||
|
||||
XCAT supports both static and dynamic groups. A static group is defined to contain a specific set of cluster nodes. A dynamic node group is one that has its members determined by specifying a selection criteria for node attributes. If a nodes attribute values match the selection criteria then it is dynamically included as a member of the group. The actual group membership will change over time as nodes have attributes set or unset. This provides flexible control over group membership by defining the attributes that define the group, rather than the specific node names that belong to the group. The selection criteria is a list of ``attr<operator>val`` pairs that can be used to determine the members of a group, (see below).
|
||||
|
||||
``Note`` : Dynamic node group support is available in xCAT version 2.3 and later.
|
||||
|
||||
In xCAT, the definition of a static group has been extended to include additional attributes that would normally be assigned to individual nodes. When a node is part of a static group definition, it can inherit the attributes assigned to the group. This feature can make it easier to define and manage cluster nodes in that you can generally assign nodes to the appropriate group and then just manage the group definition instead of multiple node definitions. This feature is not supported for dynamic groups.
|
||||
|
||||
To list all the attributes that may be set for a group definition you can run ::
|
||||
|
||||
lsdef -t group -h
|
||||
|
||||
When a node is included in one or more static groups, a particular node attribute could actually be stored in several different object definitions. It could be in the node definition itself or it could be in one or more static group definitions. The precedence for determining which value to use is to choose the attribute value specified in the node definition if it is provided. If not, then each static group that the node belongs to will be checked to see if the attribute is set. The first value that is found is the value that is used. The static groups are checked in the order that they are specified in the ``groups`` attribute of the node definition.
|
||||
|
||||
``NOTE`` : In a large cluster environment it is recommended to focus on group definitions as much as possible and avoid setting the attribute values in the individual node definition. (Of course some attribute values, such as a MAC addresses etc., are only appropriate for individual nodes.) Care must be taken to avoid confusion over which values will be inherited by the nodes.
|
||||
|
||||
Group definitions can be created using the ``mkdef`` command, changed using the ``chdef`` command, listed using the ``lsdef`` command and removed using the ``rmdef`` command.
|
||||
|
||||
Creating a static node group
|
||||
----------------------------
|
||||
|
||||
There are two basic ways to create xCAT static node groups. You can either set the ``groups`` attribute of the node definition or you can create a group definition directly.
|
||||
|
||||
You can set the ``groups`` attribute of the node definition when you are defining the node with the ``mkdef`` or ``nodeadd`` command or you can modify the attribute later using the ``chdef`` or ``nodech`` command. For example, if you want a set of nodes to be added to the group "aixnodes",you could run ``chdef`` or ``nodech`` as follows ::
|
||||
|
||||
chdef -t node -p -o node01,node02,node03 groups=aixnodes
|
||||
|
||||
or ::
|
||||
|
||||
nodech node01,node02,node03 groups=aixnodes
|
||||
|
||||
The ``-p`` (plus) option specifies that "aixnodes" be added to any existing value for the ``groups`` attribute. The ``-p`` (plus) option is not supported by ``nodech`` command.
|
||||
|
||||
The second option would be to create a new group definition directly using the ``mkdef`` command as follows ::
|
||||
|
||||
mkdef -t group -o aixnodes members="node01,node02,node03"
|
||||
|
||||
These two options will result in exactly the same definitions and attribute values being created in the xCAT database.
|
||||
|
||||
Creating a dynamic node group
|
||||
-----------------------------
|
||||
|
||||
The selection criteria for a dynamic node group is specified by providing a list of ``attr<operator>val`` pairs that can be used to determine the members of a group. The valid operators include: ``==``, ``!=``, ``=~`` and ``!~``. The ``attr`` field can be any node definition attribute returned by the ``lsdef`` command. The ``val`` field in selection criteria can be a simple sting or a regular expression. A regular expression can only be specified when using the ``=~`` or ``!~`` operators. See <TODO http://www.perl.com/doc/manual/html/pod/perlre.html> for information on the format and syntax of regular expressions.
|
||||
|
||||
Operator descriptions ::
|
||||
|
||||
== Select nodes where the attribute value is exactly this value.
|
||||
!= Select nodes where the attribute value is not this specific value.
|
||||
=~ Select nodes where the attribute value matches this regular expression.
|
||||
!~ Select nodes where the attribute value does not match this regular expression.
|
||||
|
||||
The selection criteria can be specified using one or more ``-w attr<operator>val`` options on the command line.
|
||||
|
||||
If the ``val`` field includes spaces or any other characters that will be parsed by shell then the ``attr<operator>val`` needs to be quoted.
|
||||
|
||||
For example, to create a dynamic node group called "mygroup", where the hardware control point is "hmc01" and the partition profile is not set to service ::
|
||||
|
||||
mkdef -t group -o mygroup -d -w hcp==hmc01 -w pprofile!=service
|
||||
|
||||
To create a dynamic node group called "pslesnodes", where the operating system name includes "sles" and the architecture includes "ppc" ::
|
||||
|
||||
mkdef -t group -o pslesnodes -d -w os=~sles[0-9]+ -w arch=~ppc
|
||||
|
||||
To create a dynamic node group called nonpbladenodes where the node hardware management method is not set to blade and the architecture does not include ppc ::
|
||||
|
||||
mkdef -t group -o nonpbladenodes -d -w mgt!=blade -w 'arch!~ppc'
|
||||
|
||||
|
||||
@ -1,10 +1,14 @@
|
||||
Objects
|
||||
=======
|
||||
xCAT Objects Types
|
||||
==================
|
||||
|
||||
xCAT abstracts several types of objects from the cluster information in xCAT database to represent the physical or logical entities in the cluster. Each xCAT object has a set of attributes, each attribute is mapped from a specified field of a xCAT table. The xCAT users can get cluster information and perform cluster management work through operations against the objects.
|
||||
|
||||
This section presents a brief introduction on xCAT objects, including some basic object types and xCAT commands on objects.
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
node.rst
|
||||
osimage.rst
|
||||
group.rst
|
||||
nodegroup.rst
|
||||
object_command.rst
|
||||
basic_object.rst
|
||||
|
||||
|
||||
@ -1,5 +1,120 @@
|
||||
``node``
|
||||
========
|
||||
node
|
||||
====
|
||||
|
||||
Description
|
||||
-----------
|
||||
|
||||
The physical unit in the cluster, such as lpar,virtual machine, frame, cec, hmc, switch.
|
||||
|
||||
Key Attrubutes
|
||||
--------------
|
||||
|
||||
* os:
|
||||
The operating system deployed on this node. Valid values: AIX, rhels*,rhelc*, rhas*,centos*,SL*, fedora*, sles* (where * is the version #)
|
||||
|
||||
* arch:
|
||||
The hardware architecture of this node. Valid values: x86_64, ppc64, x86, ia64.
|
||||
|
||||
* groups:
|
||||
Usually, there are a set of nodes with some attributes in common, xCAT admin can define a node group containing these nodes, so that the management task can be issued against the group instead of individual nodes. A node can be a memeber of different groups, so the value of this attributes is a comma-delimited list of groups. At least one group is required to create a node. The new created group names should not be prefixed with "__" as this token has been preserverd as the intrnal group name.
|
||||
|
||||
* mgt:
|
||||
The method to do general hardware management of the node. This attribute can be determined by the machine type of the node. Valid values: ipmi, blade, hmc, ivm, fsp, bpa, kvm, esx, rhevm.
|
||||
|
||||
* mac:
|
||||
The mac address of the network card on the node, which is connected with the installation server and can be used as the network installation device.
|
||||
|
||||
* ip:
|
||||
The IP address of the node.
|
||||
|
||||
* netboot:
|
||||
The type of network boot method for this node, determined by the OS to provision, the architecture and machine type of the node. Valid values:
|
||||
|
||||
+--------------------------+----------------------+-----------------------------------+
|
||||
| Arch and Machine Type | OS | valid netboot options |
|
||||
+==========================+======================+===================================+
|
||||
| x86, x86_64 | ALL | pxe, xnba |
|
||||
+--------------------------+----------------------+-----------------------------------+
|
||||
| ppc64 | <=rhel6, <=sles11 | yaboot |
|
||||
+--------------------------+----------------------+-----------------------------------+
|
||||
| ppc64 | >=rhels7 | grub2,grub2-http,grub2-tftp |
|
||||
+--------------------------+----------------------+-----------------------------------+
|
||||
| ppc64le NonVirtualize | ALL | petitboot |
|
||||
+--------------------------+----------------------+-----------------------------------+
|
||||
| ppc64le PowerKVM Guest | ALL | grub2,grub2-http,grub2-tftp |
|
||||
+-------------------------------------------------+-----------------------------------+
|
||||
|
||||
* postscripts:
|
||||
Comma separated list of scripts, that should be run on this node after diskfull installation or diskless boot, finish some system configuration and maintenance work. For installation of RedHat, CentOS, Fedora, the scripts will be run before the reboot. For installation of SLES, the scripts will be run after the reboot but before the init.d process.
|
||||
|
||||
* postbootscripts:
|
||||
Comma separated list of scripts, that should be run on this node as a SysV init job on the 1st reboot after installation or diskless boot, finish some system configuration and maintenance work.
|
||||
|
||||
* provmethod:
|
||||
The provisioning method for node deployment. Usually, this attribute is an ``osimage`` object name.
|
||||
|
||||
* status:
|
||||
The current status of the node, which is updated by xCAT. This value can be used to monitor the provision process. Valid values: powering-off,installing,booting/netbooting,booted.
|
||||
|
||||
Use Cases
|
||||
---------
|
||||
|
||||
* Case 1:
|
||||
There is a ppc64 lpar named "cn1", the hardware management method is "hmc", the mac of installation NIC is "ca:68:d3:ae:db:03", the ip assigned is "10.0.0.100", the network boot method is "grub2", place it into the group "all". Use the following command ::
|
||||
|
||||
mkdef -t node -o cn1 arch=ppc64 mgt=hmc mac="ca:68:d3:ae:db:03" ip="10.0.0.100" netboot="grub2" groups="all"
|
||||
|
||||
* Case 2:
|
||||
List all the node objects ::
|
||||
|
||||
nodels
|
||||
|
||||
This can also be done with ::
|
||||
|
||||
lsdef -t node
|
||||
|
||||
* Case 3:
|
||||
List the mac of object "cn1" ::
|
||||
|
||||
lsdef -t node -o cn1 -i mac
|
||||
|
||||
* Case 4:
|
||||
There is a node definition "cn1", modify its network boot method to "yaboot" ::
|
||||
|
||||
chdef -t node -o cn1 netboot=yaboot
|
||||
|
||||
* Case 5:
|
||||
There is a node definition "cn1", create a node definition "cn2" with the same attributes with "cn1", except the mac addr(ca:68:d3:ae:db:04) and ip address(10.0.0.101)
|
||||
|
||||
*step 1*: write the definition of "cn1" to a stanza file named "cn.stanza" ::
|
||||
|
||||
lsdef -z cn1 > /tmp/cn.stanza
|
||||
|
||||
The content of "/tmp/cn.stanza" will look like ::
|
||||
|
||||
# <xCAT data object stanza file>
|
||||
|
||||
cn1:
|
||||
objtype=node
|
||||
groups=all
|
||||
ip=10.0.0.100
|
||||
mac=ca:68:d3:ae:db:03
|
||||
mgt=hmc
|
||||
netboot=grub2
|
||||
|
||||
*step 2*: modify the "/tmp/cn.stanza" according to the "cn2" attributes ::
|
||||
|
||||
# <xCAT data object stanza file>
|
||||
|
||||
cn2:
|
||||
objtype=node
|
||||
groups=all
|
||||
ip=10.0.0.101
|
||||
mac=ca:68:d3:ae:db:04
|
||||
mgt=hmc
|
||||
netboot=grub2
|
||||
|
||||
*step 3*: create "cn2" definition with "cn.stanza" ::
|
||||
|
||||
cat /tmp/cn.stanza |mkdef -z
|
||||
|
||||
Node Range
|
||||
----------
|
||||
|
||||
@ -1,2 +0,0 @@
|
||||
``nodegroup``
|
||||
=============
|
||||
@ -0,0 +1,23 @@
|
||||
Object Commands
|
||||
===============
|
||||
|
||||
There are 4 commands in xCAT to manage the objects:
|
||||
|
||||
* ``mkdef`` : create object definitions
|
||||
* ``chdef`` : modify object definitions
|
||||
* ``lsdef`` : list object definitions
|
||||
* ``rmdef`` : remove object definitions
|
||||
|
||||
There are 5 commans in xCAT to manage the node objects:
|
||||
|
||||
* ``nodels`` : list the nodes and their attributes
|
||||
* ``nodeadd`` : add nodes to the xCAT cluster database
|
||||
* ``nodech`` : change nodes' attributes in the xCAT cluster database
|
||||
* ``noderm`` : removes the nodes in the noderange from all database table
|
||||
* ``nodepurge`` : automatically removes all nodes from the database and any related configurations used by the node.
|
||||
|
||||
|
||||
For details on these commands, please refer to [TODO the manpage] or run ``man <command>``.
|
||||
|
||||
|
||||
|
||||
@ -1,2 +1,94 @@
|
||||
``osimage``
|
||||
===========
|
||||
osimage
|
||||
=======
|
||||
|
||||
Description
|
||||
-----------
|
||||
|
||||
A logic definition of image which can be used to provision the node.
|
||||
|
||||
Key Attributes
|
||||
--------------
|
||||
|
||||
* imagetype:
|
||||
The type of operating system this definition represents (linux,AIX).
|
||||
|
||||
* osarch:
|
||||
The hardware architecture of the nodes this image supports. Valid values: x86_64, ppc64, ppc64le.
|
||||
|
||||
* osvers:
|
||||
The Linux distribution name and release number of the image. Valid values: rhels*,rhelc*, rhas*,centos*,SL*, fedora*, sles* (where * is the version #).
|
||||
|
||||
* pkgdir:
|
||||
The name of the directory where the copied in OS distro content are stored.
|
||||
|
||||
* pkglist:
|
||||
The fully qualified name of a file, which contains the list of packages shipped in Linux distribution ISO which will be installed on the node.
|
||||
|
||||
* otherpkgdir
|
||||
When xCAT user need to install some additional packages not shipped in Linux distribution ISO, these packages can be placed in the directory specified in this attribute. xCAT user should take care the dependency problem themselves, put all the dependency packages not shipped in Linux distribution ISO in this directory and create repository in this directory.
|
||||
|
||||
|
||||
* otherpkglist:
|
||||
The fully qualified name of a file, which contains the list of user specified additional packages not shipped in Linux distribution ISO which will be installed on the node.
|
||||
|
||||
* template:
|
||||
The fully qualified name of the template file that will be used to create the OS installer configuration file for stateful installation (e.g. kickstart for RedHat, autoyast for SLES and preseed for Ubuntu).
|
||||
|
||||
|
||||
Use Cases
|
||||
---------
|
||||
|
||||
* Case 1:
|
||||
|
||||
List all the osimage objects ::
|
||||
|
||||
lsdef -t osimage
|
||||
|
||||
* Case 2:
|
||||
|
||||
Create a osimage definition "customized-rhels7-ppc64-install-compute" based on an existed osimage "rhels7-ppc64-install-compute", the osimage "customized-rhels7-ppc64-install-compute" will inherit all the attributes of "rhels7-ppc64-install-compute" except installing the additional packages specified in the file "/tmp/otherpkg.list":
|
||||
|
||||
*step 1* : write the osimage definition "rhels7-ppc64-install-compute" to a stanza file "osimage.stanza" ::
|
||||
|
||||
lsdef -z -t osimage -o rhels7-ppc64-install-compute > /tmp/osimage.stanza
|
||||
|
||||
The content will look like ::
|
||||
|
||||
# <xCAT data object stanza file>
|
||||
|
||||
rhels7-ppc64-install-compute:
|
||||
objtype=osimage
|
||||
imagetype=linux
|
||||
osarch=ppc64
|
||||
osdistroname=rhels7-ppc64
|
||||
osname=Linux
|
||||
osvers=rhels7
|
||||
otherpkgdir=/install/post/otherpkgs/rhels7/ppc64
|
||||
pkgdir=/install/rhels7/ppc64
|
||||
pkglist=/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist
|
||||
profile=compute
|
||||
provmethod=install
|
||||
template=/opt/xcat/share/xcat/install/rh/compute.rhels7.tmpl
|
||||
|
||||
*step 2* : modify the stanza file according to the attributes of "customized-rhels7-ppc64-install-compute" ::
|
||||
|
||||
# <xCAT data object stanza file>
|
||||
|
||||
customized-rhels7-ppc64-install-compute:
|
||||
objtype=osimage
|
||||
imagetype=linux
|
||||
osarch=ppc64
|
||||
osdistroname=rhels7-ppc64
|
||||
osname=Linux
|
||||
osvers=rhels7
|
||||
otherpkglist=/tmp/otherpkg.list
|
||||
otherpkgdir=/install/post/otherpkgs/rhels7/ppc64
|
||||
pkgdir=/install/rhels7/ppc64
|
||||
pkglist=/opt/xcat/share/xcat/install/rh/compute.rhels7.pkglist
|
||||
profile=compute
|
||||
provmethod=install
|
||||
template=/opt/xcat/share/xcat/install/rh/compute.rhels7.tmpl
|
||||
|
||||
*step 3* : create the osimage "customized-rhels7-ppc64-install-compute" with the stanza file ::
|
||||
|
||||
cat /tmp/osimage.stanza |mkdef -z
|
||||
|
||||
Reference in New Issue
Block a user