xCAT/xcat-core
2
0
Fork 0
mirror of https://github.com/xcat2/xcat-core.git synced 2025-05-29 17:23:08 +00:00
Code Issues Releases Wiki Activity

add content

Browse Source
This commit is contained in:
huweihua 2015-09-01 22:04:00 -04:00
parent a127e2834d
commit cc4442402d
6 changed files with 1278 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

515
docs/source/guides/admin-guides/manage_clusters/common/deployment/cfg_partition.rst
View File

@ -1,2 +1,513 @@
Configure Disk Partition
========================
.. BEGIN_Overview
By default, xCAT will install the operating system on the first disk and with default partitions layout in the node. However, you may choose to customize the disk partitioning during the install process and define a specific disk layout. You can do this in one of two ways: '**partition definition file**' or '**partition definition script**'.
**Notes**
- 'Partition definition file' way can be used for RedHat, SLES and Ubuntu.
- 'partition definition script' way was tested only for RedHat and Ubuntu, use this feature on SLES at your own risk.
- Cause disk configuration for Ubuntu is different from RedHat, so there are some specific section for ubuntu.
.. END_Overview
.. BEGIN_partition_definition_file_Overview
You could create a customized osimage partition file, say /install/custom/my-partitions, that contains the disk partitioning definition, then associate the partition file with osimage, the nodeset command will insert the contents of this file directly into the generated autoinst configuration file that will be used by the OS installer.
.. END_partition_definition_file_Overview
.. BEGIN_partition_definition_file_content
The partition file must follow the partitioning syntax of the installer(e.g. kickstart for RedHat, AutoYaST for SLES, Preseed for Ubuntu). you could refer to the `Kickstart documentation <http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition>`_ or `Autoyast documentation <https://doc.opensuse.org/projects/autoyast/configuration.html#CreateProfile.Partitioning>`_ or `Preseed documentation <https://www.debian.org/releases/stable/i386/apbs04.html.en#preseed-partman>`_ write your own partitions layout. Meanwhile, RedHat and SuSE provides some tools that could help generate kickstart/autoyast templates, in which you could refer to the partition section for the partitions layout information:
#. RedHat:
- The file /root/anaconda-ks.cfg is a sample kickstart file created by RedHat installer during the installation process based on the options that you selected.
- system-config-kickstart is a tool with graphical interface for creating kickstart files
#. SLES
- Use yast2 autoyast in GUI or CLI mode to customize the installation options and create autoyast file
- Use yast2 clone_system to create autoyast configuration file /root/autoinst.xml to clone an existing system
#. Ubuntu
- For detailed information see the files partman-auto-recipe.txt and partman-auto-raid-recipe.txt included in the debian-installer package. Both files are also available from the debian-installer source repository. Note that the supported functionality may change between releases.
.. END_partition_definition_file_content
.. BEGIN_partition_definition_file_example_RedHat_Standard_Partitions_for_IBM_Power_machines
Here is partition definition file example for RedHat standard partition in IBM Power machines
::
# Uncomment this PReP line for IBM Power servers
#part None --fstype "PPC PReP Boot" --size 8 --ondisk sda
# Uncomment this efi line for x86_64 servers
#part /boot/efi --size 50 --ondisk /dev/sda --fstype efi
part /boot --size 256 --fstype ext4
part swap --recommended --ondisk sda
part / --size 1 --grow --fstype ext4 --ondisk sda
.. END_partition_definition_file_example_RedHat_Standard_Partitions_for_IBM_Power_machines
.. BEGIN_partition_definition_file_example_RedHat_LVM_for_IBM_Power_machines
Here is partition definition file example for RedHat LVM partition in IBM Power machines
::
# Uncomment this PReP line for IBM Power servers
#part None --fstype "PPC PReP Boot" --ondisk /dev/sda --size 8
# Uncomment this efi line for x86_64 servers
#part /boot/efi --size 50 --ondisk /dev/sda --fstype efi
part /boot --size 256 --fstype ext4 --ondisk /dev/sda
part swap --recommended --ondisk /dev/sda
part pv.01 --size 1 --grow --ondisk /dev/sda
volgroup system pv.01
logvol / --vgname=system --name=root --size 1 --grow --fstype ext4
.. END_partition_definition_file_example_RedHat_LVM_for_IBM_Power_machines
.. BEGIN_partition_definition_file_example_RedHat_RAID1_for_IBM_Power_machines
Partition definition file example for RedHat RAID1 please refer to `Configure RAID before Deploy OS <http://xcat-docs.readthedocs.org/en/latest/guides/admin-guides/manage_clusters/ppc64le/diskful/customize_image/raid_cfg.html>`_
.. END_partition_definition_file_example_RedHat_RAID1_for_IBM_Power_machines
.. BEGIN_partition_definition_file_example_SLES_Standard_Partitions_for_X86_64
Here is partition definition file example for SLES standard partition in X86_64 machines
::
<drive>
<device>/dev/sda</device>
<initialize config:type="boolean">true</initialize>
<use>all</use>
<partitions config:type="list">
<partition>
<create config:type="boolean">true</create>
<filesystem config:type="symbol">swap</filesystem>
<format config:type="boolean">true</format>
<mount>swap</mount>
<mountby config:type="symbol">path</mountby>
<partition_nr config:type="integer">1</partition_nr>
<partition_type>primary</partition_type>
<size>32G</size>
</partition>
<partition>
<create config:type="boolean">true</create>
<filesystem config:type="symbol">ext3</filesystem>
<format config:type="boolean">true</format>
<mount>/</mount>
<mountby config:type="symbol">path</mountby>
<partition_nr config:type="integer">2</partition_nr>
<partition_type>primary</partition_type>
<size>64G</size>
</partition>
</partitions>
</drive>
.. END_partition_definition_file_example_SLES_Standard_Partitions_for_X86_64
.. BEGIN_partition_definition_file_example_SLES_LVM_for_X86_64
Here is partition definition file example for SLES LVM partition in X86_64 machines
::
<drive>
<device>/dev/sda</device>
<initialize config:type="boolean">true</initialize>
<partitions config:type="list">
<partition>
<create config:type="boolean">true</create>
<crypt_fs config:type="boolean">false</crypt_fs>
<filesystem config:type="symbol">ext3</filesystem>
<format config:type="boolean">true</format>
<loop_fs config:type="boolean">false</loop_fs>
<mountby config:type="symbol">device</mountby>
<partition_id config:type="integer">65</partition_id>
<partition_nr config:type="integer">1</partition_nr>
<pool config:type="boolean">false</pool>
<raid_options/>
<resize config:type="boolean">false</resize>
<size>8M</size>
<stripes config:type="integer">1</stripes>
<stripesize config:type="integer">4</stripesize>
<subvolumes config:type="list"/>
</partition>
<partition>
<create config:type="boolean">true</create>
<crypt_fs config:type="boolean">false</crypt_fs>
<filesystem config:type="symbol">ext3</filesystem>
<format config:type="boolean">true</format>
<loop_fs config:type="boolean">false</loop_fs>
<mount>/boot</mount>
<mountby config:type="symbol">device</mountby>
<partition_id config:type="integer">131</partition_id>
<partition_nr config:type="integer">2</partition_nr>
<pool config:type="boolean">false</pool>
<raid_options/>
<resize config:type="boolean">false</resize>
<size>256M</size>
<stripes config:type="integer">1</stripes>
<stripesize config:type="integer">4</stripesize>
<subvolumes config:type="list"/>
</partition>
<partition>
<create config:type="boolean">true</create>
<crypt_fs config:type="boolean">false</crypt_fs>
<format config:type="boolean">false</format>
<loop_fs config:type="boolean">false</loop_fs>
<lvm_group>vg0</lvm_group>
<mountby config:type="symbol">device</mountby>
<partition_id config:type="integer">142</partition_id>
<partition_nr config:type="integer">3</partition_nr>
<pool config:type="boolean">false</pool>
<raid_options/>
<resize config:type="boolean">false</resize>
<size>max</size>
<stripes config:type="integer">1</stripes>
<stripesize config:type="integer">4</stripesize>
<subvolumes config:type="list"/>
</partition>
</partitions>
<pesize></pesize>
<type config:type="symbol">CT_DISK</type>
<use>all</use>
</drive>
<drive>
<device>/dev/vg0</device>
<initialize config:type="boolean">true</initialize>
<partitions config:type="list">
<partition>
<create config:type="boolean">true</create>
<crypt_fs config:type="boolean">false</crypt_fs>
<filesystem config:type="symbol">swap</filesystem>
<format config:type="boolean">true</format>
<loop_fs config:type="boolean">false</loop_fs>
<lv_name>swap</lv_name>
<mount>swap</mount>
<mountby config:type="symbol">device</mountby>
<partition_id config:type="integer">130</partition_id>
<partition_nr config:type="integer">5</partition_nr>
<pool config:type="boolean">false</pool>
<raid_options/>
<resize config:type="boolean">false</resize>
<size>auto</size>
<stripes config:type="integer">1</stripes>
<stripesize config:type="integer">4</stripesize>
<subvolumes config:type="list"/>
</partition>
<partition>
<create config:type="boolean">true</create>
<crypt_fs config:type="boolean">false</crypt_fs>
<filesystem config:type="symbol">ext3</filesystem>
<format config:type="boolean">true</format>
<loop_fs config:type="boolean">false</loop_fs>
<lv_name>root</lv_name>
<mount>/</mount>
<mountby config:type="symbol">device</mountby>
<partition_id config:type="integer">131</partition_id>
<partition_nr config:type="integer">1</partition_nr>
<pool config:type="boolean">false</pool>
<raid_options/>
<resize config:type="boolean">false</resize>
<size>max</size>
<stripes config:type="integer">1</stripes>
<stripesize config:type="integer">4</stripesize>
<subvolumes config:type="list"/>
</partition>
</partitions>
<pesize></pesize>
<type config:type="symbol">CT_LVM</type>
<use>all</use>
</drive>
.. END_partition_definition_file_example_SLES_LVM_for_X86_64
.. BEGIN_partition_definition_file_example_SLES_Standard_partition_for_ppc64
Here is partition definition file example for SLES standard partition in ppc64 machines
::
<drive>
<device>/dev/sda</device>
<initialize config:type="boolean">true</initialize>
<partitions config:type="list">
<partition>
<create config:type="boolean">true</create>
<crypt_fs config:type="boolean">false</crypt_fs>
<filesystem config:type="symbol">ext3</filesystem>
<format config:type="boolean">false</format>
<loop_fs config:type="boolean">false</loop_fs>
<mountby config:type="symbol">device</mountby>
<partition_id config:type="integer">65</partition_id>
<partition_nr config:type="integer">1</partition_nr>
<resize config:type="boolean">false</resize>
<size>auto</size>
</partition>
<partition>
<create config:type="boolean">true</create>
<crypt_fs config:type="boolean">false</crypt_fs>
<filesystem config:type="symbol">swap</filesystem>
<format config:type="boolean">true</format>
<fstopt>defaults</fstopt>
<loop_fs config:type="boolean">false</loop_fs>
<mount>swap</mount>
<mountby config:type="symbol">id</mountby>
<partition_id config:type="integer">130</partition_id>
<partition_nr config:type="integer">2</partition_nr>
<resize config:type="boolean">false</resize>
<size>auto</size>
</partition>
<partition>
<create config:type="boolean">true</create>
<crypt_fs config:type="boolean">false</crypt_fs>
<filesystem config:type="symbol">ext3</filesystem>
<format config:type="boolean">true</format>
<fstopt>acl,user_xattr</fstopt>
<loop_fs config:type="boolean">false</loop_fs>
<mount>/</mount>
<mountby config:type="symbol">id</mountby>
<partition_id config:type="integer">131</partition_id>
<partition_nr config:type="integer">3</partition_nr>
<resize config:type="boolean">false</resize>
<size>max</size>
</partition>
</partitions>
<pesize></pesize>
<type config:type="symbol">CT_DISK</type>
<use>all</use>
</drive>
.. END_partition_definition_file_example_SLES_Standard_partition_for_ppc64
.. BEGIN_partition_definition_file_example_SLES_RAID1
Partition definition file example for SLES RAID1 please refer to `Configure RAID before Deploy OS <http://xcat-docs.readthedocs.org/en/latest/guides/admin-guides/manage_clusters/ppc64le/diskful/customize_image/raid_cfg.html>`_
.. END_partition_definition_file_example_SLES_RAID1
.. BEGIN_partition_definition_file_example_Ubuntu_Standard_partition_for_PPC64le
Here is partition definition file example for Ubuntu standard partition in ppc64le machines
::
8 1 32 prep
$primary{ }
$bootable{ }
method{ prep } .
256 256 512 ext3
$primary{ }
method{ format }
format{ }
use_filesystem{ }
filesystem{ ext3 }
mountpoint{ /boot } .
64 512 300% linux-swap
method{ swap }
format{ } .
512 1024 4096 ext3
$primary{ }
method{ format }
format{ }
use_filesystem{ }
filesystem{ ext4 }
mountpoint{ / } .
100 10000 1000000000 ext3
method{ format }
format{ }
use_filesystem{ }
filesystem{ ext4 }
mountpoint{ /home } .
.. END_partition_definition_file_example_Ubuntu_Standard_partition_for_PPC64le
.. BEGIN_partition_definition_file_example_Ubuntu_Standard_partition_for_x86_64
Here is partition definition file example for Ubuntu standard partition in x86_64 machines
::
256 256 512 vfat
$primary{ }
method{ format }
format{ }
use_filesystem{ }
filesystem{ vfat }
mountpoint{ /boot/efi } .
256 256 512 ext3
$primary{ }
method{ format }
format{ }
use_filesystem{ }
filesystem{ ext3 }
mountpoint{ /boot } .
64 512 300% linux-swap
method{ swap }
format{ } .
512 1024 4096 ext3
$primary{ }
method{ format }
format{ }
use_filesystem{ }
filesystem{ ext4 }
mountpoint{ / } .
100 10000 1000000000 ext3
method{ format }
format{ }
use_filesystem{ }
filesystem{ ext4 }
mountpoint{ /home } .
.. END_partition_definition_file_example_Ubuntu_Standard_partition_for_x86_64
.. BEGIN_partition_definition_file_Associate_partition_file_with_osimage_common
Run below commands to associate the partition with the osimage
::
chdef -t osimage <osimagename> partitionfile=/install/custom/my-partitions
nodeset <nodename> osimage=<osimage>
- For Redhat, when nodeset runs and generates the /install/autoinst file for a node, it will replace the #XCAT_PARTITION_START#...#XCAT_PARTITION_END# directives from your osimage template with the contents of your custom partitionfile.
- For Ubuntu, when nodeset runs and generates the /install/autoinst file for a node, it will generate a script to write the partition configuration to /tmp/partitionfile, this script will replace the #XCA_PARTMAN_RECIPE_SCRIPT# directive in /install/autoinst/<node>.pre.
.. END_partition_definition_file_Associate_partition_file_with_osimage_common
.. BEGIN_Partition_Definition_Script_overview
Create a shell script that will be run on the node during the install process to dynamically create the disk partitioning definition. This script will be run during the OS installer %pre script on Redhat or preseed/early_command on Unbuntu execution and must write the correct partitioning definition into the file /tmp/partitionfile on the node
.. END_Partition_Definition_Script_overview
.. BEGIN_Partition_Definition_Script_Create_partition_script_content
The purpose of the partition script is to create the /tmp/partionfile that will be inserted into the kickstart/autoyast/preseed template, the script could include complex logic like select which disk to install and even configure RAID, etc
**Note**: the partition script feature is not thoroughly tested on SLES, there might be problems, use this feature on SLES at your own risk.
.. END_Partition_Definition_Script_Create_partition_script_content
.. BEGIN_Partition_Definition_Script_Create_partition_script_example_redhat_sles
Here is an example of the partition script on Redhat and SLES, the partitioning script is /install/custom/my-partitions.sh:
::
instdisk="/dev/sda"
modprobe ext4 >& /dev/null
modprobe ext4dev >& /dev/null
if grep ext4dev /proc/filesystems > /dev/null; then
FSTYPE=ext3
elif grep ext4 /proc/filesystems > /dev/null; then
FSTYPE=ext4
else
FSTYPE=ext3
fi
BOOTFSTYPE=ext3
EFIFSTYPE=vfat
if uname -r|grep ^3.*el7 > /dev/null; then
FSTYPE=xfs
BOOTFSTYPE=xfs
EFIFSTYPE=efi
fi
if [ `uname -m` = "ppc64" ]; then
echo 'part None --fstype "PPC PReP Boot" --ondisk '$instdisk' --size 8' >> /tmp/partitionfile
fi
if [ -d /sys/firmware/efi ]; then
echo 'bootloader --driveorder='$instdisk >> /tmp/partitionfile
echo 'part /boot/efi --size 50 --ondisk '$instdisk' --fstype $EFIFSTYPE' >> /tmp/partitionfile
else
echo 'bootloader' >> /tmp/partitionfile
fi
echo "part /boot --size 512 --fstype $BOOTFSTYPE --ondisk $instdisk" >> /tmp/partitionfile
echo "part swap --recommended --ondisk $instdisk" >> /tmp/partitionfile
echo "part / --size 1 --grow --ondisk $instdisk --fstype $FSTYPE" >> /tmp/partitionfile
.. END_Partition_Definition_Script_Create_partition_script_example_redhat_sles
.. BEGIN_Partition_Definition_Script_Create_partition_script_example_ubuntu
The following is an example of the partition script on Ubuntu, the partitioning script is /install/custom/my-partitions.sh:
::
if [ -d /sys/firmware/efi ]; then
echo "ubuntu-efi ::" > /tmp/partitionfile
echo " 512 512 1024 fat16" >> /tmp/partitionfile
echo ' $iflabel{ gpt } $reusemethod{ } method{ efi } format{ }' >> /tmp/partitionfile
echo " ." >> /tmp/partitionfile
else
echo "ubuntu-boot ::" > /tmp/partitionfile
echo "100 50 100 ext3" >> /tmp/partitionfile
echo ' $primary{ } $bootable{ } method{ format } format{ } use_filesystem{ } filesystem{ ext3 } mountpoint{ /boot }' >> /tmp/partitionfile
echo " ." >> /tmp/partitionfile
fi
echo "500 10000 1000000000 ext3" >> /tmp/partitionfile
echo " method{ format } format{ } use_filesystem{ } filesystem{ ext3 } mountpoint{ / }" >> /tmp/partitionfile
echo " ." >> /tmp/partitionfile
echo "2048 512 300% linux-swap" >> /tmp/partitionfile
echo " method{ swap } format{ }" >> /tmp/partitionfile
echo " ." >> /tmp/partitionfile
.. END_Partition_Definition_Script_Create_partition_script_example_ubuntu
.. BEGIN_Partition_Definition_Script_Associate_partition_script_with_osimage_common
Run below commands to associate partition script with osimage:
::
chdef -t osimage <osimagename> partitionfile='s:/install/custom/my-partitions.sh'
nodeset <nodename> osimage=<osimage>
- The "s:" preceding the filename tells nodeset that this is a script.
- For Redhat, when nodeset runs and generates the /install/autoinst file for a node, it will add the execution of the contents of this script to the %pre section of that file. The nodeset command will then replace the #XCAT_PARTITION_START#...#XCAT_PARTITION_END# directives from the osimage template file with "%include /tmp/partitionfile" to dynamically include the tmp definition file your script created.
- For Ubuntu, when nodeset runs and generates the /install/autoinst file for a node, it will replace the "#XCA_PARTMAN_RECIPE_SCRIPT#" directive and add the execution of the contents of this script to the /install/autoinst/<node>.pre, the /install/autoinst/<node>.pre script will be run in the preseed/early_command.
.. END_Partition_Definition_Script_Associate_partition_script_with_osimage_common
.. BEGIN_Partition_Disk_File_ubuntu_only
The disk file contains the name of the disks to partition in traditional, non-devfs format and delimited with space " ", for example,
::
/dev/sda /dev/sdb
If not specified, the default value will be used.
**Associate partition disk file with osimage**
::
chdef -t osimage <osimagename> -p partitionfile='d:/install/custom/partitiondisk'
nodeset <nodename> osimage=<osimage>
- the 'd:' preceding the filename tells nodeset that this is a partition disk file.
- For Ubuntu, when nodeset runs and generates the /install/autoinst file for a node, it will generate a script to write the content of the partition disk file to /tmp/boot_disk, this context to run the script will replace the #XCA_PARTMAN_DISK_SCRIPT# directive in /install/autoinst/<node>.pre.
.. END_Partition_Disk_File_ubuntu_only
.. BEGIN_Partition_Disk_Script_ubuntu_only
The disk script contains a script to generate a partitioning disk file named "/tmp/boot_disk". for example,
::
rm /tmp/devs-with-boot 2>/dev/null || true;
for d in $(list-devices partition); do
mkdir -p /tmp/mymount;
rc=0;
mount $d /tmp/mymount || rc=$?;
if [[ $rc -eq 0 ]]; then
[[ -d /tmp/mymount/boot ]] && echo $d >>/tmp/devs-with-boot;
umount /tmp/mymount;
fi
done;
if [[ -e /tmp/devs-with-boot ]]; then
head -n1 /tmp/devs-with-boot | egrep -o '\S+[^0-9]' > /tmp/boot_disk;
rm /tmp/devs-with-boot 2>/dev/null || true;
else
DEV=`ls /dev/disk/by-path/* -l | egrep -o '/dev.*[s|h|v]d[^0-9]$' | sort -t : -k 1 -k 2 -k 3 -k 4 -k 5 -k 6 -k 7 -k 8 -g | head -n1 | egrep -o '[s|h|v]d.*$'`;
if [[ "$DEV" == "" ]]; then DEV="sda"; fi;
echo "/dev/$DEV" > /tmp/boot_disk;
fi;
If not specified, the default value will be used.
**Associate partition disk script with osimage**
::
chdef -t osimage <osimagename> -p partitionfile='s:d:/install/custom/partitiondiskscript'
nodeset <nodename> osimage=<osimage>
- the 's:' prefix tells nodeset that is a script, the 's:d:' preceding the filename tells nodeset that this is a script to generate the partition disk file.
- For Ubuntu, when nodeset runs and generates the /install/autoinst file for a node, this context to run the script will replace the #XCA_PARTMAN_DISK_SCRIPT# directive in /install/autoinst/<node>.pre.
.. END_Partition_Disk_Script_ubuntu_only
.. BEGIN_Additional_preseed_configuration_file_ubuntu_only
To support other specific partition methods such as RAID or LVM in Ubuntu, some additional preseed configuration entries should be specified.
If using file way, 'c:<the absolute path of the additional preseed config file>', the additional preseed config file contains the additional preseed entries in "d-i ..." syntax. When "nodeset", the #XCA_PARTMAN_ADDITIONAL_CFG# directive in /install/autoinst/<node> will be replaced with content of the config file, an example:
::
d-i partman-auto/method string raid
d-i partman-md/confirm boolean true
If not specified, the default value will be used.
.. END_Additional_preseed_configuration_file_ubuntu_only
.. BEGIN_Additional_preseed_configuration_script_ubuntu_only
To support other specific partition methods such as RAID or LVM in Ubuntu, some additional preseed configuration entries should be specified.
If using script way, 's:c:<the absolute path of the additional preseed config script>', the additional preseed config script is a script to set the preseed values with "debconf-set". When "nodeset", the #XCA_PARTMAN_ADDITIONAL_CONFIG_SCRIPT# directive in /install/autoinst/<node>.pre will be replaced with the content of the script, an example:
::
debconf-set partman-auto/method string raid
debconf-set partman-md/confirm boolean true
If not specified, the default value will be used.
.. END_Additional_preseed_configuration_script_ubuntu_only

128
docs/source/guides/admin-guides/manage_clusters/common/deployment/cfg_second_adapter.rst
View File

@ -1,2 +1,130 @@
Configure Secondary Network Adapter
===================================
Introduction
------------
The **nics** table and the **confignics** postscript can be used to automatically configure additional **ethernet** and **Infiniband** adapters on nodes as they are being deployed. ("Additional adapters" means adapters other than the primary adapter that the node is being installed/booted over.)
The way the confignics postscript decides what IP address to give the secondary adapter is by checking the nics table, in which the nic configuration information is stored.
To use the nics table and confignics postscript to define a secondary adapter on one or more nodes, follow these steps:
Define configuration information for the Secondary Adapters in the nics table
-----------------------------------------------------------------------------
There are 3 ways to complete this operation.
**First way is use command line input. below is a example**
::
[root@ls21n01 ~]# mkdef cn1 groups=all nicips.eth1="11.1.89.7|12.1.89.7" nicnetworks.eth1="net11|net12" nictypes.eth1="Ethernet"
1 object definitions have been created or modified.
[root@ls21n01 ~]# chdef cn1 nicips.eth2="13.1.89.7|14.1.89.7" nicnetworks.eth2="net13|net14" nictypes.eth2="Ethernet"
1 object definitions have been created or modified.
**Second way is using stanza file**
prepare your stanza file <filename>.stanza. the content of <filename>.stanza like below:
::
# <xCAT data object stanza file>
cn1:
objtype=node
arch=x86_64
groups=kvm,vm,all
nichostnamesuffixes.eth1=-eth1-1|-eth1-2
nichostnamesuffixes.eth2=-eth2-1|-eth2-2
nicips.eth1=11.1.89.7|12.1.89.7
nicips.eth2=13.1.89.7|14.1.89.7
nicnetworks.eth1=net11|net12
nicnetworks.eth2=net13|net14
nictypes.eth1=Ethernet
nictypes.eth2=Ethernet
define configuration information by <filename>.stanza
::
cat <filename>.stanza | mkdef -z
**Third way is use 'tabedit' to edit the nics table directly**
The 'tabedit' command opens the specified table in the user's editor(such as VI), allows user to edit any text, and then writes changes back to the database table. But it's tedious and error prone, so don't recommended this way. if using this way, notices the **nicips**, **nictypes** and **nicnetworks** attributes are required.
Here is a sample nics table content:
::
[root@ls21n01 ~]# tabdump nics
#node,nicips,nichostnamesuffixes,nictypes,niccustomscripts,nicnetworks,nicaliases,comments,disable
"cn1","eth1!11.1.89.7|12.1.89.7,eth2!13.1.89.7|14.1.89.7","eth1!-eth1-1|-eth1-2,eth2!-eth2-1|-eth2-2,"eth1!Ethernet,eth2!Ethernet",,"eth1!net11|net12,eth2!net13|net14",,,
After you have define configuration information by any way above, you can run below command to put configuration information into /etc/hosts:
::
makehosts cn1
Then /etc/hosts will looks like:
::
11.1.89.7 cn1-eth1-1 cn1-eth1-1.ppd.pok.ibm.com
12.1.89.7 cn1-eth1-2 cn1-eth1-2.ppd.pok.ibm.com
13.1.89.7 cn1-eth2-1 cn1-eth2-1.ppd.pok.ibm.com
14.1.89.7 cn1-eth2-2 cn1-eth2-2.ppd.pok.ibm.com
Add confignics into the node's postscripts list
-----------------------------------------------
Using below command to add confignics into the node's postscripts list
::
chdef cn1 -p postscripts=confignics
By default, confignics does not configure the install nic. if need, using flag "-s" to allow the install nic to be configured.
::
chdef cn1 -p prostscripts="confignics -s"
Option "-s" write the install nic's information into configuration file for persistance. All install nic's data defined in nics table will be written also.
Add network object into the networks table
------------------------------------------
The nicnetworks attribute only defined the network object name which used by the ip address. Other information about the network should be define in the networks table. Can use tabedit to add/ modify the networks objects.
::
#netname,net,mask,mgtifname,gateway,dhcpserver,tftpserver,nameservers,ntpservers,logservers,dynamicrange,staticrange,staticrangeincrement,nodehostname,ddnsdomain,vlanid,domain,comments,disable
...
"net11", "11.1.89.0", "255.255.255.0", "eth1",,,,,,,,,,,,,,,
"net12", "12.1.89.0", "255.255.255.0", "eth1",,,,,,,,,,,,,,,
"net13", "13.1.89.0", "255.255.255.0", "eth2",,,,,,,,,,,,,,,
"net14", "14.1.89.0", "255.255.255.0", "eth2",,,,,,,,,,,,,,,
Option -r to remove the undefined NICS
---------------------------------------
If the compute node's nics were configured by confignics, and the nics configuration changed in the nics table, can use "confignics -r" to remove the undefined nics. For example: On the compute node the eth0, eth1 and eth2 were configured
::
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:14:5e:d9:6c:e6
...
eth1 Link encap:Ethernet HWaddr 00:14:5e:d9:6c:e7
...
eth2 Link encap:Ethernet HWaddr 00:14:5e:d9:6c:e8
...
Delete the eth2 definition in nics table with chdef command. Run
::
updatenode <noderange> -P "confignics -r" to remove the undefined eth2 on the compute node.
The complete result is:
::
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:14:5e:d9:6c:e6
...
eth1 Link encap:Ethernet HWaddr 00:14:5e:d9:6c:e7
...
Deleting the install nic will import some strange problems. So confignics -r can not delete the install nic.

45
docs/source/guides/admin-guides/manage_clusters/common/deployment/create_img.rst
View File

@ -1,4 +1,47 @@
Select or Create an osimage Definition
======================================
Run copycds to create image
Before creating image by xCAT, distro media should be prepared ahead. That can be ISOs or DVDs.
XCAT use 'copycds' command to create image which will be available to install nodes. 'copycds' command copies the contents of distro from media to /install/<os>/<arch> on management node.
If using an ISO, copy it to (or NFS mount it on) the management node, and then run:
::
copycds <path>/<specific-distro>.iso
If using a DVD, put it in the DVD drive of the management node and run:
::
copycds /dev/<dvd-drive-name>
The 'copycds' command automatically creates several osimage defintions in the database that can be used for node deployment.
To see the list of osimages, run
::
lsdef -t osimage
To see the attributes of a particular osimage, run
::
lsdef -t osimage <osimage-name>
Initially, some attributes of osimage is assigned to default value by xCAT, they all can work correctly, cause the files or templates invoked by those attributes are shipped with xCAT by default. If need to customize those attribute, refer to next section "Customize osimage".
**[Below tips maybe helpful for you]**
**[Tips 1]**
If this is the same distro version as what your management node used, create a .repo file in /etc/yum.repos.d with content similar to:
::
[local-<os>-<arch>]
name=xCAT local <os> <version>
baseurl=file:/install/<os>/<arch>
enabled=1
gpgcheck=0
In this way, if you need install some additional RPMs into your MN later, you can simply install them by yum. Or if you are installing a software on your MN that depends some RPMs from the this disto, those RPMs will be found and installed automatically.
**[Tips 2]**
If need to change osimage name to your favorite name, below statement maybe help:
::
lsdef -t osimage -z rhels6.2-x86_64-install-compute | sed 's/^[^ ]\+:/mycomputeimage:/' | mkdef -z

31
docs/source/guides/admin-guides/manage_clusters/common/deployment/deploy_os.rst
View File

@ -1,10 +1,35 @@
Initialize the Compute for Deployment
=====================================
nodeset
XCAT use '**nodeset**' command to associate a specific image to a node which will be installed with this image.
::
nodeset <nodename> osimage=<osimage>
There are more attributes of nodeset used for some specific purpose or specific machines, for example:
* **runimage**: If you would like to run a task after deployment, you can define that task with this attribute.
* **runcmd**: This instructs the node to boot to the xCAT nbfs environment and proceed to configure BMC for basic remote access. This causes the IP, netmask, gateway, username, and password to be programmed according to the configuration table.
* **shell**: This instructs tho node to boot to the xCAT genesis environment, and present a shell prompt on console. The node will also be able to be sshed into and have utilities such as wget, tftp, scp, nfs, and cifs. It will have storage drivers available for many common systems.
Choose such additional attribute of nodeset according to your requirement, if want to get more informantion about nodeset, refer to nodeset's man page.
Start the OS Deployment
=======================
rsetboot <node> net
rpower <node> on
Start the deployment involves two key operations. One is setup node boot from network, another is reboot ndoe:
For Power machine, those two operations can be completed by one command '**rnetboot**',
::
rnetboot <node>
But for x server, those two operations need two independent commands.
Set x server boot from network, run
::
rsetboot <node> net
Reboot x server:
::
rpower <node> reset

125
docs/source/guides/admin-guides/manage_clusters/common/deployment/driver_update_disk.rst
View File

@ -1,2 +1,123 @@
Using Driver Update Disk
========================
.. BEGIN_Overview
During the installing or netbooting of a node, the drivers in the initrd will be used to drive the devices like network cards and IO devices to perform the installation/netbooting tasks. But sometimes the drivers for the new devices were not included in the default initrd shipped by Red Hat or Suse. A solution is to inject the new drivers into the initrd to drive the new device during the installation/netbooting process.
Generally there are two approaches to inject the new drivers: **Driver Update Disk** and **Drive RPM package**.
A "**Driver Update Disk**" is media which contains the drivers, firmware and related configuration files for certain devices. The driver update disk is always supplied by the vendor of the device. One driver update disk can contain multiple drivers for different OS releases and different hardware architectures. Red Hat and Suse have different driver update disk formats.
The '**Driver RPM Package**' is the rpm package which includes the drivers and firmware for the specific devices. The Driver RPM is the rpm package which is shipped by the Vendor of the device for a new device or a new kernel version.
xCAT supports both. But for '**Driver RPM Package**' is only supported in xCAT 2.8 and later.
No matter which approach chosen, there are two steps to make new drivers work. one is locate the new driver's path, another is inject the new drivers into the initrd.
.. END_Overview
.. BEGIN_locate_driver_for_DUD
There are two approaches for xCAT to find the driver disk (pick one):
- Specify the location of the driver disk in the osimage object (*This is ONLY supported in 2.8 and later*)
The value for the 'driverupdatesrc' attribute is a comma separated driver disk list. The tag 'dud' must be specified before the full path of 'driver update disk' to specify the type of the file:
::
chdef -t osimage <osimagename> driverupdatesrc=dud:<full path of driver disk>
- Put the driver update disk in the directory <installroot>/driverdisk/<os>/<arch> (*e.g. /install/driverdisk/sles11.1/x86_64*). During the running of the 'genimage', 'geninitrd' or 'nodeset' command, xCAT will look for driver update disks in the directory <installroot>/driverdisk/<os>/<arch>.
.. END_locate_driver_for_DUD
.. BEGIN_locate_driver_for_RPM
The Driver RPM packages must be specified in the osimage object.
Three attributes of osimage object can be used to specify the Driver RPM location and Driver names. If you want to load new drivers in the initrd, the '**netdrivers**' attribute must be set. And one or both of the '**driverupdatesrc**' and '**osupdatename**' attributes must be set. If both of 'driverupdatesrc' and 'osupdatename' are set, the drivers in the 'driverupdatesrc' have higher priority.
- netdrivers - comma separated driver names that need to be injected into the initrd. The postfix '.ko' can be ignored.
The 'netdrivers' attribute must be set to specify the new driver list. If you want to load all the drivers from the driver rpms, use the keyword allupdate. Another keyword for the netdrivers attribute is updateonly, which means only the drivers located in the original initrd will be added to the newly built initrd from the driver rpms. This is useful to reduce the size of the new built initrd when the distro is updated, since there are many more drivers in the new kernel rpm than in the original initrd. Examples:
::
chdef -t osimage <osimagename> netdrivers=megaraid_sas.ko,igb.ko
chdef -t osimage <osimagename> netdrivers=allupdate
chdef -t osimage <osimagename> netdrivers=updateonly,igb.ko,new.ko
- driverupdatesrc - comma separated driver rpm packages (full path should be specified)
A tag named 'rpm' can be specified before the full path of the rpm to specify the file type. The tag is optional since the default format is 'rpm' if no tag is specified. Example:
::
chdef -t osimage <osimagename> driverupdatesrc=rpm:<full path of driver disk1>,rpm:<full path of driver disk2>
- osupdatename - comma separated 'osdistroupdate' objects. Each 'osdistroupdate' object specifies a Linux distro update.
When geninitrd is run, 'kernel-*.rpm' will be searched in the osdistroupdate.dirpath to get all the rpm packages and then those rpms will be searched for drivers. Example:
::
mkdef -t osdistroupdate update1 dirpath=/install/<os>/<arch>
chdef -t osimage <osimagename> osupdatename=update1
If 'osupdatename' is specified, the kernel shipped with the 'osupdatename' will be used to load the newly built initrd, then only the drivers matching the new kernel will be kept in the newly built initrd. If trying to use the 'osupdatename', the 'allupdate' or 'updateonly' should be added in the 'netdrivers' attribute, or all the necessary driver names for the new kernel need to be added in the 'netdrivers' attribute. Otherwise the new drivers for the new kernel will be missed in newly built initrd.
.. END_locate_driver_for_RPM
.. BEGIN_inject_into_initrd__for_diskfull_for_DUD
- If specifying the driver disk location in the osimage, there are two ways to inject drivers:
The first way is:
::
nodeset <noderange> osimage=<osimagename>
The Second way is:
::
geninitrd <osimagename>
nodeset <noderange> osimage=<osimagename> --noupdateinitrd
**Note:** 'geninitrd' + 'nodeset --noupdateinitrd' is useful when you need to run nodeset frequently for a diskful node. 'geninitrd' only needs be run once to rebuild the initrd and 'nodeset --noupdateinitrd' will not touch the initrd and kernel in /tftpboot/xcat/osimage/<osimage name>/.
- If putting the driver disk in <installroot>/driverdisk/<os>/<arch>:
Running 'nodeset <nodenrage>' in anyway will load the driver disk
.. END_inject_into_initrd__for_diskfull_for_DUD
.. BEGIN__inject_into_initrd__for_diskfull_for_RPM
There are two ways to inject drivers, one is:
::
nodeset <noderange> osimage=<osimagename> [--ignorekernelchk]
Another is:
::
geninitrd <osimagename> [--ignorekernelchk]
nodeset <noderange> osimage=<osimagename> --noupdateinitrd
**Note:** 'geninitrd' + 'nodeset --noupdateinitrd' is useful when you need to run nodeset frequently for diskful nodes. 'geninitrd' only needs to be run once to rebuild the initrd and 'nodeset --noupdateinitrd' will not touch the initrd and kernel in /tftpboot/xcat/osimage/<osimage name>/.
The option '--ignorekernelchk' is used to skip the kernel version checking when injecting drivers from osimage.driverupdatesrc. To use this flag, you should make sure the drivers in the driver rpms are usable for the target kernel.
.. END_inject_into_initrd__for_diskfull_for_RPM
.. BEGIN_inject_into_initrd__for_diskless_for_DUD
- If specifying the driver disk location in the osimage
Run the below command:
::
genimage <osimagename>
- If putting the driver disk in <installroot>/driverdisk/<os>/<arch>:
Running 'genimage' in anyway will load the driver disk
.. END_inject_into_initrd__for_diskless_for_DUD
.. BEGIN_inject_into_initrd__for_diskless_for_RPM
Run the below command:
::
genimage <osimagename> [--ignorekernelchk]
The option '--ignorekernelchk' is used to skip the kernel version checking when injecting drivers from osimage.driverupdatesrc. To use this flag, you should make sure the drivers in the driver rpms are usable for the target kernel.
.. END_inject_into_initrd__for_diskless_for_RPM
.. BEGIN_node
- If the drivers from the driver disk or driver rpm are not already part of the installed or booted system, it's necessary to add the rpm packages for the drivers to the .pkglist or .otherpkglist of the osimage object to install them in the system.
- If a driver rpm needs to be loaded, the osimage object must be used for the 'nodeset' and 'genimage' command, instead of the older style profile approach.
- Both a Driver disk and a Driver rpm can be loaded in one 'nodeset' or 'genimage' invocation.
.. END_node

442
docs/source/guides/admin-guides/manage_clusters/common/deployment/raid_cfg.rst
View File

@ -1,2 +1,444 @@
Configure RAID before Deploy OS
===============================
Overview
--------
This section describes how to use xCAT to deploy diskful nodes with RAID1 setup, and the procedure for RAID1 maintainence activities such as replacing a failed disk.
All the examples in this section are based on three configuration scenarios:
- RHEL6 on a system p machine with two SCSI disks sda and sdb
- RHEL6 on a system p machine with two SAS disks and multipath configuration.
- SLES 11 SP1 on a system p machine with two SCSI disks sda and sdb
If you are not using the configuration scenarios listed above, you may need to modify some of the steps in this documentation to make it work in your environment.
Deploy Diskful Nodes with RAID1 Setup on RedHat
-----------------------------------------------
xCAT provides two sample kickstart template files with the RAID1 settings, /opt/xcat/share/xcat/install/rh/service.raid1.rhel6.ppc64.tmpl is for the configuration scenario #1 listed above and /opt/xcat/share/xcat/install/rh/service.raid1.multipath.rhel6.ppc64.tmpl is for the configuration scenario #2 listed above. You can customize the template file and put it under /install/custom/install/<platform>/ if the default one does not match your requirements.
Here is the RAID1 partitioning section in service.raid1.rhel6.ppc64.tmpl:
::
#Full RAID 1 Sample
part None --fstype "PPC PReP Boot" --size 8 --ondisk sda --asprimary
part None --fstype "PPC PReP Boot" --size 8 --ondisk sdb --asprimary
part raid.01 --size 200 --fstype ext4 --ondisk sda
part raid.02 --size 200 --fstype ext4 --ondisk sdb
raid /boot --level 1 --device md0 raid.01 raid.02
part raid.11 --size 1024 --ondisk sda
part raid.12 --size 1024 --ondisk sdb
raid swap --level 1 --device md1 raid.11 raid.12
part raid.21 --size 1 --fstype ext4 --grow --ondisk sda
part raid.22 --size 1 --fstype ext4 --grow --ondisk sdb
raid / --level 1 --device md2 raid.21 raid.22
And here is the RAID1 partitioning section in service.raid1.multipath.rhel6.ppc64.tmpl
::
#Full RAID 1 Sample
part None --fstype "PPC PReP Boot" --size 8 --ondisk mpatha --asprimary
part None --fstype "PPC PReP Boot" --size 8 --ondisk mpathb --asprimary
part raid.01 --size 200 --fstype ext4 --ondisk mpatha
part raid.02 --size 200 --fstype ext4 --ondisk mpathb
raid /boot --level 1 --device md0 raid.01 raid.02
part raid.11 --size 1024 --ondisk mpatha
part raid.12 --size 1024 --ondisk mpathb
raid swap --level 1 --device md1 raid.11 raid.12
part raid.21 --size 1 --fstype ext4 --grow --ondisk mpatha
part raid.22 --size 1 --fstype ext4 --grow --ondisk mpathb
raid / --level 1 --device md2 raid.21 raid.22
The samples above created one PReP partition, one 200MB /boot partition and one / partition on sda/sda and mpatha/mpathb. If you want to use different partitioning scheme in your cluster, modify this RAID1 section in the kickstart template file accordingly.
After the diskful nodes are up and running, you can check the RAID1 settings with the following commands:
Mount command shows the /dev/mdx devices are mounted to various file systems, the /dev/mdx indicates that the RAID is being used on this node.
::
[root@server ~]# mount
/dev/md2 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/md0 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
The file /proc/mdstat includes the RAID devices status on the system, here is an example of /proc/mdstat in the non-multipath environment:
::
[root@server ~]# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sda5[0] sdb5[1]
19706812 blocks super 1.1 [2/2] [UU]
bitmap: 1/1 pages [64KB], 65536KB chunk
md1 : active raid1 sda2[0] sdb2[1]
1048568 blocks super 1.1 [2/2] [UU]
md0 : active raid1 sda3[0] sdb3[1]
204788 blocks super 1.0 [2/2] [UU]
unused devices: <none>
On the system with multipath configuration, the /proc/mdstat looks like:
::
[root@server ~]# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 dm-11[0] dm-6[1]
291703676 blocks super 1.1 [2/2] [UU]
bitmap: 1/1 pages [64KB], 65536KB chunk
md1 : active raid1 dm-8[0] dm-3[1]
1048568 blocks super 1.1 [2/2] [UU]
md0 : active raid1 dm-9[0] dm-4[1]
204788 blocks super 1.0 [2/2] [UU]
unused devices: <none>
The command mdadm can query the detailed configuration for the RAID partitions:
::
mdadm --detail /dev/md2
Deploy Diskful Nodes with RAID1 Setup on SLES
---------------------------------------------
xCAT provides one sample autoyast template files with the RAID1 settings /opt/xcat/share/xcat/install/sles/service.raid1.sles11.tmpl. You can customize the template file and put it under /install/custom/install/<platform>/ if the default one does not match your requirements.
Here is the RAID1 partitioning section in service.raid1.sles11.tmpl:
::
<partitioning config:type="list">
<drive>
<device>/dev/sda</device>
<partitions config:type="list">
<partition>
<format config:type="boolean">false</format>
<partition_id config:type="integer">65</partition_id>
<partition_nr config:type="integer">1</partition_nr>
<partition_type>primary</partition_type>
<size>24M</size>
</partition>
<partition>
<format config:type="boolean">false</format>
<partition_id config:type="integer">253</partition_id>
<partition_nr config:type="integer">2</partition_nr>
<raid_name>/dev/md0</raid_name>
<raid_type>raid</raid_type>
<size>2G</size>
</partition>
<partition>
<format config:type="boolean">false</format>
<partition_id config:type="integer">253</partition_id>
<partition_nr config:type="integer">3</partition_nr>
<raid_name>/dev/md1</raid_name>
<raid_type>raid</raid_type>
<size>max</size>
</partition>
</partitions>
<use>all</use>
</drive>
<drive>
<device>/dev/sdb</device>
<partitions config:type="list">
<partition>
<format config:type="boolean">false</format>
<partition_id config:type="integer">131</partition_id>
<partition_nr config:type="integer">1</partition_nr>
<partition_type>primary</partition_type>
<size>24M</size>
</partition>
<partition>
<format config:type="boolean">false</format>
<partition_id config:type="integer">253</partition_id>
<partition_nr config:type="integer">2</partition_nr>
<raid_name>/dev/md0</raid_name>
<raid_type>raid</raid_type>
<size>2G</size>
</partition>
<partition>
<format config:type="boolean">false</format>
<partition_id config:type="integer">253</partition_id>
<partition_nr config:type="integer">3</partition_nr>
<raid_name>/dev/md1</raid_name>
<raid_type>raid</raid_type>
<size>max</size>
</partition>
</partitions>
<use>all</use>
</drive>
<drive>
<device>/dev/md</device>
<partitions config:type="list">
<partition>
<filesystem config:type="symbol">reiser</filesystem>
<format config:type="boolean">true</format>
<mount>swap</mount>
<partition_id config:type="integer">131</partition_id>
<partition_nr config:type="integer">0</partition_nr>
<raid_options>
<chunk_size>4</chunk_size>
<parity_algorithm>left-asymmetric</parity_algorithm>
<raid_type>raid1</raid_type>
</raid_options>
</partition>
<partition>
<filesystem config:type="symbol">reiser</filesystem>
<format config:type="boolean">true</format>
<mount>/</mount>
<partition_id config:type="integer">131</partition_id>
<partition_nr config:type="integer">1</partition_nr>
<raid_options>
<chunk_size>4</chunk_size>
<parity_algorithm>left-asymmetric</parity_algorithm>
<raid_type>raid1</raid_type>
</raid_options>
</partition>
</partitions>
<use>all</use>
</drive>
</partitioning>
The samples above created one 24MB PReP partition on each disk, one 2GB mirroed swap partition and one mirroed / partition uses all the disk space. If you want to use different partitioning scheme in your cluster, modify this RAID1 section in the autoyast template file accordingly.
Since the PReP partition can not be mirroed between the two disks, some additional postinstall commands should be run to make the second disk bootable, here the the commands needed to make the second disk bootable:
::
# Set the second disk to be bootable for RAID1 setup
parted -s /dev/sdb mkfs 1 fat16
parted /dev/sdb set 1 type 6
parted /dev/sdb set 1 boot on
dd if=/dev/sda1 of=/dev/sdb1
bootlist -m normal sda sdb
The procedure listed above has been added to the file /opt/xcat/share/xcat/install/scripts/post.sles11.raid1 to make it be automated. The autoyast template file service.raid1.sles11.tmpl will include the content of post.sles11.raid1, so no manual steps are needed here.
After the diskful nodes are up and running, you can check the RAID1 settings with the following commands:
Mount command shows the /dev/mdx devices are mounted to various file systems, the /dev/mdx indicates that the RAID is being used on this node.
::
server:~ # mount
/dev/md1 on / type reiserfs (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
debugfs on /sys/kernel/debug type debugfs (rw)
devtmpfs on /dev type devtmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,mode=1777)
devpts on /dev/pts type devpts (rw,mode=0620,gid=5)
The file /proc/mdstat includes the RAID devices status on the system, here is an example of /proc/mdstat:
::
server:~ # cat /proc/mdstat
Personalities : [raid1] [raid0] [raid10] [raid6] [raid5] [raid4]
md0 : active (auto-read-only) raid1 sda2[0] sdb2[1]
2104500 blocks super 1.0 [2/2] [UU]
bitmap: 0/1 pages [0KB], 128KB chunk
md1 : active raid1 sda3[0] sdb3[1]
18828108 blocks super 1.0 [2/2] [UU]
bitmap: 0/9 pages [0KB], 64KB chunk
unused devices: <none>
The command mdadm can query the detailed configuration for the RAID partitions:
::
mdadm --detail /dev/md1
Disk Replacement Procedure
--------------------------
If any one disk fails in the RAID1 arrary, do not panic. Follow the procedure listed below to replace the failed disk and you will be fine.
Faulty disks should appear marked with an (F) if you look at /proc/mdstat:
::
[root@server ~]# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 dm-11[0](F) dm-6[1]
291703676 blocks super 1.1 [2/1] [_U]
bitmap: 1/1 pages [64KB], 65536KB chunk
md1 : active raid1 dm-8[0](F) dm-3[1]
1048568 blocks super 1.1 [2/1] [_U]
md0 : active raid1 dm-9[0](F) dm-4[1]
204788 blocks super 1.0 [2/1] [_U]
unused devices: <none>
We can see that the first disk is broken because all the RAID partitions on this disk are marked as (F).
Remove the failed disk from RAID arrary
---------------------------------------
mdadm is the command that can be used to query and manage the RAID arrays on Linux. To remove the failed disk from RAID array, use the command:
::
mdadm --manage /dev/mdx --remove /dev/xxx
Where the /dev/mdx are the RAID partitions listed in /proc/mdstat file, such as md0, md1 and md2; the /dev/xxx are the backend devices like dm-11, dm-8 and dm-9 in the multipath configuration and sda5, sda3 and sda2 in the non-multipath configuration.
Here is the example of removing failed disk from the RAID1 array in the non-multipath configuration:
::
mdadm --manage /dev/md0 --remove /dev/sda3
mdadm --manage /dev/md1 --remove /dev/sda2
mdadm --manage /dev/md2 --remove /dev/sda5
Here is the example of removing failed disk from the RAID1 array in the multipath configuration:
::
mdadm --manage /dev/md0 --remove /dev/dm-9
mdadm --manage /dev/md1 --remove /dev/dm-8
mdadm --manage /dev/md2 --remove /dev/dm-11
After the failed disk is removed from the RAID1 array, the partitions on the failed disk will be removed from /proc/mdstat and the "mdadm --detail" output also.
::
[root@server ~]# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 dm-6[1]
291703676 blocks super 1.1 [2/1] [_U]
bitmap: 1/1 pages [64KB], 65536KB chunk
md1 : active raid1 dm-3[1]
1048568 blocks super 1.1 [2/1] [_U]
md0 : active raid1 dm-4[1]
204788 blocks super 1.0 [2/1] [_U]
unused devices: <none>
[root@server ~]# mdadm --detail /dev/md0
/dev/md0:
Version : 1.0
Creation Time : Tue Jul 19 02:39:03 2011
Raid Level : raid1
Array Size : 204788 (200.02 MiB 209.70 MB)
Used Dev Size : 204788 (200.02 MiB 209.70 MB)
Raid Devices : 2
Total Devices : 1
Persistence : Superblock is persistent
Update Time : Wed Jul 20 02:00:04 2011
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0
Name : c250f17c01ap01:0 (local to host c250f17c01ap01)
UUID : eba4d8ad:8f08f231:3c60e20f:1f929144
Events : 26
Number Major Minor RaidDevice State
0 0 0 0 removed
1 253 4 1 active sync /dev/dm-4
Replace the disk
----------------
Depends on the hot swap capability, you may simply unplug the disk and replace with a new one if the hot swap is supported; otherwise, you will need to power off the machine and replace the disk and the power on the machine.
Create partitions on the new disk
The first thing we must do now is to create the exact same partitioning as on the new disk. We can do this with one simple command:
::
sfdisk -d /dev/<good_disk> | sfdisk /dev/<new_disk>
For the non-mulipath configuration, here is an example:
::
sfdisk -d /dev/sdb | sfdisk /dev/sda
For the multipath configuration, here is an example:
::
sfdisk -d /dev/dm-1 | sfdisk /dev/dm-0
If you got error message "sfdisk: I don't like these partitions - nothing changed.", you can add "--force" option to the sfdisk command:
::
sfdisk -d /dev/sdb | sfdisk /dev/sda --force
You can run
::
fdisk -l
to check if both hard drives have the same partitioning now.
Add the new disk into the RAID1 array
-------------------------------------
After the partitions are created on the new disk, you can use command
::
mdadm --manage /dev/mdx --add /dev/xxx
to add the new disk to the RAID1 array. Where the /dev/mdx are the RAID partitions like md0, md1 and md2; the /dev/xxx are the backend devices like dm-11, dm-8 and dm-9 in the multipath configuration and sda5, sda3 and sda2 in the non-multipath configuration.
Here is an example for the non-multipath configuration:
::
mdadm --manage /dev/md0 --add /dev/sda3
mdadm --manage /dev/md1 --add /dev/sda2
mdadm --manage /dev/md2 --add /dev/sda5
Here is an example for the multipath configuration:
::
mdadm --manage /dev/md0 --add /dev/dm-9
mdadm --manage /dev/md1 --add /dev/dm-8
mdadm --manage /dev/md2 --add /dev/dm-11
All done! You can have a cup of coffee to watch the fully automatic reconstruction running...
While the RAID1 array is reconstructing, you will see some progress information in /proc/mdstat:
::
[root@server raid1]# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 dm-11[0] dm-6[1]
291703676 blocks super 1.1 [2/1] [_U]
[>....................] recovery = 0.7% (2103744/291703676) finish=86.2min speed=55960K/sec
bitmap: 1/1 pages [64KB], 65536KB chunk
md1 : active raid1 dm-8[0] dm-3[1]
1048568 blocks super 1.1 [2/1] [_U]
[=============>.......] recovery = 65.1% (683904/1048568) finish=0.1min speed=48850K/sec
md0 : active raid1 dm-9[0] dm-4[1]
204788 blocks super 1.0 [2/1] [_U]
[===================>.] recovery = 96.5% (198016/204788) finish=0.0min speed=14144K/sec
unused devices: <none>
After the reconstruction is done, the /proc/mdstat becomes like:
::
[root@server ~]# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 dm-11[0] dm-6[1]
291703676 blocks super 1.1 [2/2] [UU]
bitmap: 1/1 pages [64KB], 65536KB chunk
md1 : active raid1 dm-8[0] dm-3[1]
1048568 blocks super 1.1 [2/2] [UU]
md0 : active raid1 dm-9[0] dm-4[1]
204788 blocks super 1.0 [2/2] [UU]
unused devices: <none>
Make the new disk bootable
--------------------------
If the new disk does not have a PReP partition or the PReP partition has some problem, it will not be bootable, here is an example on how to make the new disk bootable, you may need to substitute the device name with your own values.
**RedHat:**
::
mkofboot .b /dev/sda
bootlist -m normal sda sdb
**SLES:**
::
parted -s /dev/sda mkfs 1 fat16
parted /dev/sda set 1 type 6
parted /dev/sda set 1 boot on
dd if=/dev/sdb1 of=/dev/sda1
bootlist -m normal sda sdb