From d4c5d579eafd696333f15abf1c9c895bf69fc4cd Mon Sep 17 00:00:00 2001 From: Bin Xu Date: Mon, 23 Oct 2017 14:15:57 +0800 Subject: [PATCH 1/8] stateless SN document refine --- .../manage_clusters/common/deployment/enable_localdisk.rst | 2 +- .../manage_clusters/ppc64le/diskless/localdisk.rst | 4 +--- xCAT-server/lib/xcat/plugins/getpartition.pm | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/common/deployment/enable_localdisk.rst b/docs/source/guides/admin-guides/manage_clusters/common/deployment/enable_localdisk.rst index 5608abe72..eae674382 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/deployment/enable_localdisk.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/deployment/enable_localdisk.rst @@ -80,6 +80,6 @@ For the files/directories that you would like xCAT to store on the local disk, a ``Note``: you do not need to specify the swap space in the litefile table. Just putting it in the partitionfile config file is enough. -Add an entry in policy table to permit the running of the ``getpartitioin`` command from the node :: +Add an entry in policy table to permit the running of the ``getpartition`` command from the node :: chtab priority=7.1 policy.commands=getpartition policy.rule=allow diff --git a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/localdisk.rst b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/localdisk.rst index 54922a50c..6d5d7a35c 100644 --- a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/localdisk.rst +++ b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/localdisk.rst @@ -1,6 +1,4 @@ .. include:: ../../../common/deployment/enable_localdisk.rst ``Note``: - * `localdisk` feature won't syncronize the files/directories defined in `litefile` table from diskless image to local disk at the node boot time. It might casue issue to the application which depends on some of those directories. For example, the ``httpd`` service cannot be started if ``/var/log/`` is defined in `litefile` table. To work around this, you may copy the required contents to local disk and restart service manually at the first time. - - * To keep the contents on local disk after you use ``enablepart=yes`` to do partitioin, make sure to set ``enablepart=no`` in partitioin configuration file after the node is booted. \ No newline at end of file + * To keep the contents on local disk after you using ``enablepart=yes`` to do the disk partitioning, make sure to set ``enablepart=no`` in partition configuration file after the node is booted. \ No newline at end of file diff --git a/xCAT-server/lib/xcat/plugins/getpartition.pm b/xCAT-server/lib/xcat/plugins/getpartition.pm index 9ea728417..de136693e 100644 --- a/xCAT-server/lib/xcat/plugins/getpartition.pm +++ b/xCAT-server/lib/xcat/plugins/getpartition.pm @@ -3,7 +3,7 @@ =head1 xCAT plugin package to handle getpartition command - Generally, the getpartitioin command is called from the stateless + Generally, the getpartition command is called from the stateless node during the booting period to get the partition configureation infomation to part the hard disk on the stateless node and to manage the local disk space for statelite and swap space. From 1409955b2867138a0b6e58884eaff2d08aac4504 Mon Sep 17 00:00:00 2001 From: Bin Xu Date: Tue, 24 Oct 2017 16:07:05 +0800 Subject: [PATCH 2/8] Fix #3955 Refine the stateless SN and add the content with localdisk --- .../hierarchy/provision/diskless_sn.rst | 206 ++++++++++++------ .../common/deployment/enable_localdisk.rst | 2 + .../diskless/customize_image/localdisk.rst | 3 + .../ppc64le/diskless/localdisk.rst | 4 - 4 files changed, 144 insertions(+), 71 deletions(-) delete mode 100644 docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/localdisk.rst diff --git a/docs/source/advanced/hierarchy/provision/diskless_sn.rst b/docs/source/advanced/hierarchy/provision/diskless_sn.rst index 717d3ee18..f51834f73 100644 --- a/docs/source/advanced/hierarchy/provision/diskless_sn.rst +++ b/docs/source/advanced/hierarchy/provision/diskless_sn.rst @@ -18,47 +18,57 @@ Build the Service Node Diskless Image This section assumes you can build the stateless image on the management node because the Service Nodes are the same OS and architecture as the management node. If this is not the case, you need to build the image on a machine that matches the Service Node's OS architecture. -* Create an osimage definition. When you run ``copycds``, xCAT will create a Service Node osimage definitions for that distribution. For a stateless - Service Node, use the ``*-netboot-service`` definition. :: +* Create an osimage definition. When you run ``copycds``, xCAT will only create a Service Node stateful osimage definitions for that distribution. For a stateless Service Node, you may create it from a stateless Compute Node osimage definition. :: - # lsdef -t osimage | grep -i service - rhels6.4-ppc64-install-service (osimage) - rhels6.4-ppc64-netboot-service (osimage) <================ - rhels6.4-ppc64-statelite-service (osimage) + lsdef -t osimage | grep -i netboot + rhels7.3-ppc64le-netboot-compute (osimage) - # lsdef -t osimage -l rhels6.3-ppc64-netboot-service - Object name: rhels6.3-ppc64-netboot-service - exlist=/opt/xcat/share/xcat/netboot/rh/service.exlist + mkdef -t osimage -o rhels7.3-ppc64le-netboot-service \ + --template rhels7.3-ppc64le-netboot-compute \ + profile=service provmethod=netboot postscripts=servicenode \ + exlist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.exlist \ + otherpkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.otherpkgs.pkglist \ + pkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.pkglist \ + postinstall=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.postinstall \ + rootimgdir=/install/netboot/rhels7.3/ppc64le/service + + lsdef -t osimage -l rhels7.3-ppc64le-netboot-service + Object name: rhels7.3-ppc64le-netboot-service + exlist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.exlist imagetype=linux - osarch=ppc64 - osdistroname=rhels6.3-ppc64 + osarch=ppc64le + osdistroname=rhels7.3-ppc64le osname=Linux - osvers=rhels6.3 - otherpkgdir=/install/post/otherpkgs/rhels6.3/ppc64 - otherpkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels6.ppc64.otherpkgs.pkglist - pkgdir=/install/rhels6.3/ppc64 - pkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels6.ppc64.pkglist - postinstall=/opt/xcat/share/xcat/netboot/rh/service.rhels6.ppc64.postinstall + osvers=rhels7.3 + otherpkgdir=/install/post/otherpkgs/rhels7.3/ppc64le + otherpkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.otherpkgs.pkglist + pkgdir=/install/rhels7.3/ppc64le + pkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.pkglist + postinstall=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.postinstall + postscripts=servicenode profile=service provmethod=netboot - rootimgdir=/install/netboot/rhels6.3/ppc64/service + rootimgdir=/install/netboot/rhels7.3/ppc64le/service * You can check the Service Node packaging to see if it has all the rpms you require. We ship a basic requirements lists that will create a fully functional Service Node. However, you may want to customize your service node by adding additional operating system packages or modifying the files excluded by the exclude list. View the files referenced by the osimage pkglist, otherpkglist and exlist attributes: :: cd /opt/xcat/share/xcat/netboot/rh/ - view service.rhels6.ppc64.pkglist - view service.rhels6.ppc64.otherpkgs.pkglist - view service.exlist + view service.rhels7.ppc64le.pkglist + view service.rhels7.ppc64le.otherpkgs.pkglist + view service.rhels7.ppc64le.exlist If you would like to change any of these files, copy them to a custom directory. This can be any directory you choose, but we recommend that you keep it /install somewhere. A good location is something like - ``/install/custom/netboot//service``. Make sure that your + ``/install/custom/netboot/``. Make sure that your ``otherpkgs.pkglist`` file as an entry for :: xcat/xcat-core/xCATsn + xcat/xcat-dep/rh7/ppc64le/conserver-xcat + xcat/xcat-dep/rh7/ppc64le/perl-Net-Telnet + xcat/xcat-dep/rh7/ppc64le/perl-Expect This is required to install the xCAT Service Node function into your image. @@ -68,30 +78,31 @@ This section assumes you can build the stateless image on the management node be :: - cp /opt/xcat/share/xcat/netboot/rh/service.rhels6.ppc64.postinstall - /install/custom/netboot/rh - vi /install/custom/netboot/rh - # uncomment the sample fstab lines and change as needed: - proc /proc proc rw 0 0 - sysfs /sys sysfs rw 0 0 - devpts /dev/pts devpts rw,gid=5,mode=620 0 0 - service_x86_64 / tmpfs rw 0 1 - none /tmp tmpfs defaults,size=10m 0 2 - none /var/tmp tmpfs defaults,size=10m 0 2 + cp /opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.postinstall \ + /install/custom/netboot/rhels7.3-ppc64le-netboot-service/my.postinstall + + vi /install/custom/netboot/rhels7.3-ppc64le-netboot-service/my.postinstall + # uncomment the sample fstab lines and change as needed: + proc /proc proc rw 0 0 + sysfs /sys sysfs rw 0 0 + devpts /dev/pts devpts rw,gid=5,mode=620 0 0 + service_ppc64le / tmpfs rw 0 1 + none /tmp tmpfs defaults,size=10m 0 2 + none /var/tmp tmpfs defaults,size=10m 0 2 After modifying the files, you will need to update the osimage definition to reference these files. We recommend creating a new osimage definition for your custom image: :: - lsdef -t osimage -l rhels6.3-ppc64-netboot-service -z > /tmp/myservice.def + lsdef -t osimage -l rhels7.3-ppc64le-netboot-service -z > /tmp/myservice.def vi /tmp/myservice.def - # change the name of the osimage definition - # change any attributes that now need to reference your custom files - # change the rootimgdir attribute replacing 'service' - with a name to match your new osimage definition + # change the name of the osimage definition + # change any attributes that now need to reference your custom files + # change the rootimgdir attribute replacing 'service' with a name to match your new osimage definition + cat /tmp/msyservice.def | mkdef -z - While you are here, if you'd like, you can do the same for your compute node + While you are here, if you'd like, you can do the same for your Service Node images, creating custom files and new custom osimage definitions as you need to. @@ -100,10 +111,10 @@ This section assumes you can build the stateless image on the management node be * If you downloaded xCAT to your management node for installation, place a copy of your xcat-core and xcat-dep in your otherpkgdir directory :: - lsdef -t osimage -o rhels6.3-ppc64-netboot-service -i otherpkgdir - Object name: rhels6.3-ppc64-netboot-service - otherpkgdir=/install/post/otherpkgs/rhels6.3/ppc64 - cd /install/post/otherpkgs/rhels6.3/ppc64 + lsdef -t osimage -o rhels7.3-ppc64le-netboot-service -i otherpkgdir + Object name: rhels7.3-ppc64le-netboot-service + otherpkgdir=/install/post/otherpkgs/rhels7.3/ppc64le + cd /install/post/otherpkgs/rhels7.3/ppc64le mkdir xcat cd xcat cp -Rp /xcat-core @@ -117,11 +128,11 @@ This section assumes you can build the stateless image on the management node be :: - lsdef -t osimage -o rhels6.3-ppc64-netboot-service -i otherpkgdir - Object name: rhels6.3-ppc64-netboot-service - otherpkgdir=/install/post/otherpkgs/rhels6.3/ppc64 - - cd /install/post/otherpkgs/rhels6.3/ppc64 + lsdef -t osimage -o rhels7.3-ppc64le-netboot-service -i otherpkgdir + Object name: rhels7.3-ppc64le-netboot-service + otherpkgdir=/install/post/otherpkgs/rhels7.3/ppc64le + + cd /install/post/otherpkgs/rhels7.3/ppc64le mkdir xcat cd xcat @@ -135,40 +146,110 @@ This section assumes you can build the stateless image on the management node be :: - genimage rhels6.3-ppc64-netboot-service + genimage rhels7.3-ppc64le-netboot-service * Prevent DHCP from starting up until xcatd has had a chance to configure it: :: - chroot /install/netboot/rhels6.3/ppc64/service/rootimg chkconfig dhcpd off - chroot /install/netboot/rhels6.3/ppc64/service/rootimg chkconfig dhcrelay off + chroot /install/netboot/rhels7.3/ppc64le/service/rootimg chkconfig dhcpd off + chroot /install/netboot/rhels7.3/ppc64le/service/rootimg chkconfig dhcrelay off * IF using NFS hybrid mode, export /install read-only in Service Node image: :: - cd /install/netboot/rhels6.3/ppc64/service/rootimg/etc + cd /install/netboot/rhels7.3/ppc64le/service/rootimg/etc echo '/install *(ro,no_root_squash,sync,fsid=13)' >exports * Pack the image for your osimage definition: :: - packimage rhels6.3-ppc64-netboot-service + packimage rhels7.3-ppc64le-netboot-service * Set the node status to ready for netboot using your osimage definition and your 'service' nodegroup: :: - nodeset service osimage=rhels6.3-ppc64-netboot-service + nodeset service osimage=rhels7.3-ppc64le-netboot-service * To diskless boot the Service Nodes :: - rnetboot service + rsetboot service + +Monitor install and boot +------------------------ + +:: + + wcons service # make sure DISPLAY is set to your X server/VNC or + rcons # or do rcons for each node + tail -f /var/log/messages + + +Enable localdisk for stateless Service Node (Optional) +------------------------------------------------------ + +If you want, your can leverage local disk to contain some directories during the +stateless nodes running. For Service Node, it is recommended to put below directories +on local disk. :: + + #/install (Not required when using shared /install directory) + #/tftpboot (Not required when using shared /tftpboot directory) + /var/log + /tmp + +The following section explains how to accomplish this. + +* Change the Service Node osimage definition to enable ``localdisk`` + +:: + + #create a partition file to partition and mount the disk + export OSIMAGE=rhels7.3-ppc64le-netboot-service + cat< /install/custom/netboot/$OSIMAGE/partitionfile + enable=yes + enablepart=yes + + [disk] + dev=/dev/sda + clear=yes + parts=10,50 + + [localspace] + dev=/dev/sda2 + fstype=ext4 + + [swapspace] + dev=/dev/sda1 + EOF + + #add the partition file to Service Node osimage definition and configure ``policy`` table + chdef -t osimage -o $OSIMAGE partitionfile=/install/custom/netboot/$OSIMAGE/partitionfile + chtab priority=7.1 policy.commands=getpartition policy.rule=allow + + #define files or directories which are required to be put on local disk + #chtab litefile.image=$OSIMAGE litefile.file=/install/ litefile.options=localdisk + #chtab litefile.image=$OSIMAGE litefile.file=/tftpboot/ litefile.options=localdisk + chtab litefile.image=$OSIMAGE litefile.file=/var/log/ litefile.options=localdisk + chtab litefile.image=$OSIMAGE litefile.file=/tmp/ litefile.options=localdisk + +* Run image generation and repacking for your osimage definition: + + :: + + genimage rhels7.3-ppc64le-netboot-service + packimage rhels7.3-ppc64le-netboot-service + + +Note: ``enablepart=yes`` in partition file will make partition on the locak disk at every boot time. If you want to keep the contents +on local disk at next boot, remmber changing to ``enablepart=no`` after the first time provisioning. + +For more information on ``localdisk`` option, refer to :ref:`setup_localdisk_label` Update Service Node Stateless Image ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -182,10 +263,10 @@ To update the xCAT software in the image at a later time: :: - genimage rhels6.3-ppc64-netboot-service - packimage rhels6.3-ppc64-netboot-service - nodeset service osimage=rhels6.3-ppc64-netboot-service - rnetboot service + genimage rhels7.3-ppc64le-netboot-service + packimage rhels7.3-ppc64le-netboot-service + nodeset service osimage=rhels7.3-ppc64le-netboot-service + rsetboot service Note: The Service Nodes are set up as NFS-root servers for the compute nodes. Any time changes are made to any compute image on the mgmt node it will be @@ -193,12 +274,3 @@ necessary to sync all changes to all Service Nodes. In our case the ``/install`` directory is mounted on the servicenodes, so the update to the compute node image is automatically available. -Monitor install and boot ------------------------- - -:: - - wcons service # make sure DISPLAY is set to your X server/VNC or - rcons # or do rcons for each node - tail -f /var/log/messages - diff --git a/docs/source/guides/admin-guides/manage_clusters/common/deployment/enable_localdisk.rst b/docs/source/guides/admin-guides/manage_clusters/common/deployment/enable_localdisk.rst index eae674382..cd6e7d730 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/deployment/enable_localdisk.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/deployment/enable_localdisk.rst @@ -1,3 +1,5 @@ +.. _setup_localdisk_label: + Enabling the localdisk Option ----------------------------- diff --git a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/localdisk.rst b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/localdisk.rst index 52115d5ef..df0b3e202 100644 --- a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/localdisk.rst +++ b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/localdisk.rst @@ -1 +1,4 @@ .. include:: ../../../common/deployment/enable_localdisk.rst + +``Note``: + * To keep the contents on local disk after you using ``enablepart=yes`` to do the disk partitioning, make sure to set ``enablepart=no`` in partition configuration file after the node is booted. diff --git a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/localdisk.rst b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/localdisk.rst deleted file mode 100644 index 6d5d7a35c..000000000 --- a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/localdisk.rst +++ /dev/null @@ -1,4 +0,0 @@ -.. include:: ../../../common/deployment/enable_localdisk.rst - -``Note``: - * To keep the contents on local disk after you using ``enablepart=yes`` to do the disk partitioning, make sure to set ``enablepart=no`` in partition configuration file after the node is booted. \ No newline at end of file From bae354316e6511fabfc24f5415a9efbb0f01b829 Mon Sep 17 00:00:00 2001 From: Matt Ezell Date: Tue, 31 Oct 2017 21:13:20 -0400 Subject: [PATCH 3/8] Quiet some debug messages during discovery --- perl-xCAT/xCAT/MacMap.pm | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/perl-xCAT/xCAT/MacMap.pm b/perl-xCAT/xCAT/MacMap.pm index e5cd444c6..8b87f11a2 100644 --- a/perl-xCAT/xCAT/MacMap.pm +++ b/perl-xCAT/xCAT/MacMap.pm @@ -512,7 +512,7 @@ sub refresh_table { my $ntype = $typehash->{$entry->{node}}->[0]->{nodetype}; if ( (($discover_switch) and ( $ntype ne "switch")) or ( !($discover_switch) and ( $ntype eq "switch")) ){ - xCAT::MsgUtils->message("S", "refresh_table: skip $entry->{node} and $entry->{switch}, $discover_switch , $ntype\n"); + xCAT::MsgUtils->trace(0, "d", "refresh_table: skip node=$entry->{node} switch=$entry->{switch} discover_switch=$discover_switch nodetype=$ntype\n"); next; } if (defined($entry->{switch}) and $entry->{switch} ne "" and defined($entry->{port}) and $entry->{port} ne "") { @@ -561,7 +561,7 @@ sub refresh_table { $self->refresh_switch($parent, $community, $entry->{switch}); my $runstop = time; my $diffduration = $runstop - $runstart; - xCAT::MsgUtils->message("S", "refresh_switch $entry->{switch} ElapsedTime:$diffduration sec"); + xCAT::MsgUtils->trace(0, ($diffduration > 10) ? "w" : "d", "refresh_switch $entry->{switch} ElapsedTime:$diffduration sec"); exit(0); } close($parent); @@ -612,9 +612,9 @@ sub walkoid { if ($session->{ErrorStr}) { unless ($namedargs{silentfail}) { if ($namedargs{ciscowarn}) { - xCAT::MsgUtils->message("S", "Error communicating with " . $session->{DestHost} . " (First attempt at indexing by VLAN failed, ensure that the switch has the vlan configured such that it appears in 'show vlan'): " . $session->{ErrorStr}); + xCAT::MsgUtils->message("S", "Error communicating with switch " . $session->{DestHost} . " (First attempt at indexing by VLAN failed, ensure that the switch has the vlan configured such that it appears in 'show vlan'): " . $session->{ErrorStr}); } else { - xCAT::MsgUtils->message("S", "Error communicating with " . $session->{DestHost} . ": " . $session->{ErrorStr}); + xCAT::MsgUtils->message("S", "Error communicating with switch " . $session->{DestHost} . ": " . $session->{ErrorStr}); } } if ($switch) { @@ -832,7 +832,7 @@ sub refresh_switch { if ($self->{switchparmhash}->{$switch}->{switchtype} eq 'onie'){ my $bridgetoifmap = walkoid($session, '.1.3.6.1.2.1.17.1.4.1.2', ciscowarn => $iscisco, verbose => $self->{show_verbose_info}, switch => $switch, callback => $self->{callback}); # Good for all switches if (not ref $bridgetoifmap or !keys %{$bridgetoifmap}) { - xCAT::MsgUtils->message("S", "Error communicating with " . $session->{DestHost} . ": failed to get a valid response to BRIDGE-MIB request"); + xCAT::MsgUtils->message("S", "Error communicating with switch " . $session->{DestHost} . ": failed to get a valid response to BRIDGE-MIB request"); $self->{macinfo}->{$switch}->{ErrorStr} = "Failed to get a valid response to BRIDGE-MIB request"; return; } @@ -915,7 +915,7 @@ sub refresh_switch { } my $bridgetoifmap = walkoid($session, '.1.3.6.1.2.1.17.1.4.1.2', ciscowarn => $iscisco, verbose => $self->{show_verbose_info}, switch => $switch, callback => $self->{callback}); # Good for all switches if (not ref $bridgetoifmap or !keys %{$bridgetoifmap}) { - xCAT::MsgUtils->message("S", "Error communicating with " . $session->{DestHost} . ": failed to get a valid response to BRIDGE-MIB request"); + xCAT::MsgUtils->message("S", "Error communicating with switch " . $session->{DestHost} . ": failed to get a valid response to BRIDGE-MIB request"); if ($self->{collect_mac_info}) { $self->{macinfo}->{$switch}->{ErrorStr} = "Failed to get a valid response to BRIDGE-MIB request"; } @@ -928,7 +928,7 @@ sub refresh_switch { $mactoindexmap = walkoid($session, '.1.3.6.1.2.1.17.4.3.1.2', ciscowarn => $iscisco, verbose => $self->{show_verbose_info}, switch => $switch, callback => $self->{callback}); } #Ok, time to process the data if (not ref $mactoindexmap or !keys %{$mactoindexmap}) { - xCAT::MsgUtils->message("S", "Error communicating with " . $session->{DestHost} . ": Unable to get MAC entries via either BRIDGE or Q-BRIDE MIB"); + xCAT::MsgUtils->message("S", "Error communicating with switch " . $session->{DestHost} . ": Unable to get MAC entries via either BRIDGE or Q-BRIDE MIB"); if ($self->{collect_mac_info}) { $self->{macinfo}->{$switch}->{ErrorStr} = "Unable to get MAC entries via either BRIDGE or Q-BRIDE MIB"; } From 6a2d713e2936b194491d4dde6b394d0d0495855d Mon Sep 17 00:00:00 2001 From: Bin Xu Date: Wed, 1 Nov 2017 22:35:36 +0800 Subject: [PATCH 4/8] adjust the old document structure for stateless sn --- .../hierarchy/provision/diskless_sn.rst | 167 +++++++++++------- 1 file changed, 102 insertions(+), 65 deletions(-) diff --git a/docs/source/advanced/hierarchy/provision/diskless_sn.rst b/docs/source/advanced/hierarchy/provision/diskless_sn.rst index f51834f73..173c45c73 100644 --- a/docs/source/advanced/hierarchy/provision/diskless_sn.rst +++ b/docs/source/advanced/hierarchy/provision/diskless_sn.rst @@ -18,39 +18,47 @@ Build the Service Node Diskless Image This section assumes you can build the stateless image on the management node because the Service Nodes are the same OS and architecture as the management node. If this is not the case, you need to build the image on a machine that matches the Service Node's OS architecture. -* Create an osimage definition. When you run ``copycds``, xCAT will only create a Service Node stateful osimage definitions for that distribution. For a stateless Service Node, you may create it from a stateless Compute Node osimage definition. :: +* Create an osimage definition. + +When you run ``copycds``, xCAT will only create a Service Node stateful osimage definitions for that distribution. For a stateless Service Node osimage, you may create it from a stateless Compute Node osimage definition. :: lsdef -t osimage | grep -i netboot rhels7.3-ppc64le-netboot-compute (osimage) mkdef -t osimage -o rhels7.3-ppc64le-netboot-service \ --template rhels7.3-ppc64le-netboot-compute \ - profile=service provmethod=netboot postscripts=servicenode \ - exlist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.exlist \ - otherpkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.otherpkgs.pkglist \ - pkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.pkglist \ - postinstall=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.postinstall \ - rootimgdir=/install/netboot/rhels7.3/ppc64le/service + profile=service provmethod=netboot postscripts=servicenode - lsdef -t osimage -l rhels7.3-ppc64le-netboot-service + lsdef -t osimage rhels7.3-ppc64le-netboot-service Object name: rhels7.3-ppc64le-netboot-service - exlist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.exlist + exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.exlist imagetype=linux osarch=ppc64le osdistroname=rhels7.3-ppc64le osname=Linux osvers=rhels7.3 otherpkgdir=/install/post/otherpkgs/rhels7.3/ppc64le - otherpkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.otherpkgs.pkglist pkgdir=/install/rhels7.3/ppc64le - pkglist=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.pkglist - postinstall=/opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.postinstall + pkglist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.pkglist + postinstall=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.ppc64le.postinstall postscripts=servicenode profile=service provmethod=netboot - rootimgdir=/install/netboot/rhels7.3/ppc64le/service + rootimgdir=/install/netboot/rhels7.3/ppc64le/compute -* You can check the Service Node packaging to see if it has all the rpms you require. We ship a basic requirements lists that will create a fully functional Service Node. However, you may want to customize your service node by adding additional operating system packages or modifying the files excluded by the exclude list. View the files referenced by the osimage pkglist, otherpkglist and exlist attributes: :: +* Configure mandatory attributes for Service Node osimage definition. + + The following attributes must be modified to suitable value: :: + + exlist + otherpkglist + pkglist + postinstall + rootimgdir + + 1, Create the exlist, pkglist and otherpkglist file. + + xCAT ships a basic requirements lists that will create a fully functional Service Node. However, you may want to customize your service node by adding additional operating system packages or modifying the files excluded by the exclude list. Check the below files to see if it meets your needs. :: cd /opt/xcat/share/xcat/netboot/rh/ view service.rhels7.ppc64le.pkglist @@ -59,29 +67,40 @@ This section assumes you can build the stateless image on the management node be If you would like to change any of these files, copy them to a custom directory. This can be any directory you choose, but we recommend that you - keep it /install somewhere. A good location is something like - ``/install/custom/netboot/``. Make sure that your - ``otherpkgs.pkglist`` file as an entry for + keep it /install somewhere. A good location is something like ``/install/custom/netboot/``. :: - xcat/xcat-core/xCATsn - xcat/xcat-dep/rh7/ppc64le/conserver-xcat - xcat/xcat-dep/rh7/ppc64le/perl-Net-Telnet - xcat/xcat-dep/rh7/ppc64le/perl-Expect + export OSIMAGE_DIR=/install/custom/netboot/rhels7.3-ppc64le-netboot-service - This is required to install the xCAT Service Node function into your image. + mkdir -p $OSIMAGE_DIR - You may also choose to create an appropriate /etc/fstab file in your - Service Node image. Copy the script referenced by the postinstall - attribute to your directory and modify it as you would like: + cp /opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.pkglist $OSIMAGE_DIR + cp /opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.otherpkgs.pkglist $OSIMAGE_DIR + cp /opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.exlist $OSIMAGE_DIR + + And make sure that your ``otherpkgs.pkglist`` file has the following entries: :: - cp /opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.postinstall \ - /install/custom/netboot/rhels7.3-ppc64le-netboot-service/my.postinstall + xcat/xcat-dep///xCATsn + xcat/xcat-dep///conserver-xcat + xcat/xcat-dep///perl-Net-Telnet + xcat/xcat-dep///perl-Expect - vi /install/custom/netboot/rhels7.3-ppc64le-netboot-service/my.postinstall + These are required to install the xCAT Service Node function into your image. + + 2, Create the postinstall script. + + xCAT ships a default postinstall script for stateless Service Node. You may also choose to create an appropriate /etc/fstab file in your + Service Node image. : + + :: + + export OSIMAGE_DIR=/install/custom/netboot/rhels7.3-ppc64le-netboot-service + cp /opt/xcat/share/xcat/netboot/rh/service.rhels7.ppc64le.postinstall $OSIMAGE_DIR + + vi $OSIMAGE_DIR/service.rhels7.ppc64le.postinstall # uncomment the sample fstab lines and change as needed: proc /proc proc rw 0 0 sysfs /sys sysfs rw 0 0 @@ -90,17 +109,36 @@ This section assumes you can build the stateless image on the management node be none /tmp tmpfs defaults,size=10m 0 2 none /var/tmp tmpfs defaults,size=10m 0 2 - After modifying the files, you will need to update the osimage definition to - reference these files. We recommend creating a new osimage definition for - your custom image: :: + 3, Modify the Service Node osimage definition with given attributes. - lsdef -t osimage -l rhels7.3-ppc64le-netboot-service -z > /tmp/myservice.def - vi /tmp/myservice.def - # change the name of the osimage definition - # change any attributes that now need to reference your custom files - # change the rootimgdir attribute replacing 'service' with a name to match your new osimage definition + :: + + export OSIMAGE_DIR=/install/custom/netboot/rhels7.3-ppc64le-netboot-service + chdef -t osimage -o rhels7.3-ppc64le-netboot-service \ + exlist=$OSIMAGE_DIR/service.rhels7.ppc64le.exlist \ + otherpkglist=$OSIMAGE_DIR/service.rhels7.ppc64le.otherpkgs.pkglist \ + pkglist=$OSIMAGE_DIR/service.rhels7.ppc64le.pkglist \ + postinstall=$OSIMAGE_DIR/service.rhels7.ppc64le.postinstall \ + rootimgdir=$OSIMAGE_DIR/service + + lsdef -t osimage -l rhels7.3-ppc64le-netboot-service + Object name: rhels7.3-ppc64le-netboot-service + exlist=/install/custom/netboot/rhels7.3-ppc64le-netboot-service/service.rhels7.ppc64le.exlist + imagetype=linux + osarch=ppc64le + osdistroname=rhels7.3-ppc64le + osname=Linux + osvers=rhels7.3 + otherpkgdir=/install/post/otherpkgs/rhels7.3/ppc64le + otherpkglist=/install/custom/netboot/rhels7.3-ppc64le-netboot-service/service.rhels7.ppc64le.otherpkgs.pkglist + pkgdir=/install/rhels7.3/ppc64le + pkglist=/install/custom/netboot/rhels7.3-ppc64le-netboot-service/service.rhels7.ppc64le.pkglist + postinstall=/install/custom/netboot/rhels7.3-ppc64le-netboot-service/service.rhels7.ppc64le.postinstall + postscripts=servicenode + profile=service + provmethod=netboot + rootimgdir=/install/custom/netboot/rhels7.3-ppc64le-netboot-service/service - cat /tmp/msyservice.def | mkdef -z While you are here, if you'd like, you can do the same for your Service Node images, creating custom files and new custom osimage definitions as you need @@ -108,8 +146,10 @@ This section assumes you can build the stateless image on the management node be * Make your xCAT software available for otherpkgs processing -* If you downloaded xCAT to your management node for installation, place a - copy of your xcat-core and xcat-dep in your otherpkgdir directory :: + Option 1: + + If you downloaded xCAT to your management node for installation, place a + copy of your ``xcat-core`` and ``xcat-dep`` in your ``otherpkgdir`` directory :: lsdef -t osimage -o rhels7.3-ppc64le-netboot-service -i otherpkgdir Object name: rhels7.3-ppc64le-netboot-service @@ -120,7 +160,9 @@ This section assumes you can build the stateless image on the management node be cp -Rp /xcat-core cp -Rp /xcat-dep -* If you installed your management node directly from the online + Option 2: + + If you installed your management node directly from the online repository, you will need to download the ``xcat-core`` and ``xcat-dep`` tarballs - From http://xcat.org/download.html, download the ``xcat-core`` and ``xcat-dep`` tarball files. @@ -152,14 +194,16 @@ This section assumes you can build the stateless image on the management node be :: - chroot /install/netboot/rhels7.3/ppc64le/service/rootimg chkconfig dhcpd off - chroot /install/netboot/rhels7.3/ppc64le/service/rootimg chkconfig dhcrelay off + export OSIMAGE_ROOT=/install/custom/netboot/rhels7.3-ppc64le-netboot-service/service + chroot $OSIMAGE_ROOT/rootimg chkconfig dhcpd off + chroot $OSIMAGE_ROOT/rootimg chkconfig dhcrelay off * IF using NFS hybrid mode, export /install read-only in Service Node image: :: - cd /install/netboot/rhels7.3/ppc64le/service/rootimg/etc + export OSIMAGE_ROOT=/install/custom/netboot/rhels7.3-ppc64le-netboot-service/service + cd $OSIMAGE_ROOT/rootimg/etc echo '/install *(ro,no_root_squash,sync,fsid=13)' >exports * Pack the image for your osimage definition: @@ -168,26 +212,19 @@ This section assumes you can build the stateless image on the management node be packimage rhels7.3-ppc64le-netboot-service -* Set the node status to ready for netboot using your osimage definition and - your 'service' nodegroup: - - :: - - nodeset service osimage=rhels7.3-ppc64le-netboot-service - -* To diskless boot the Service Nodes - - :: - - rsetboot service - -Monitor install and boot +Install Service Nodes ------------------------ -:: + :: - wcons service # make sure DISPLAY is set to your X server/VNC or - rcons # or do rcons for each node + rinstall service osimage=rhels7.3-ppc64le-netboot-service + + Watch the installation progress using either wcons or rcons: + + :: + + wcons service # make sure DISPLAY is set to your X server/VNC or + rcons tail -f /var/log/messages @@ -195,7 +232,8 @@ Enable localdisk for stateless Service Node (Optional) ------------------------------------------------------ If you want, your can leverage local disk to contain some directories during the -stateless nodes running. For Service Node, it is recommended to put below directories +stateless nodes running. And you can customize the osimage definition to achieve it. +For Service Node, it is recommended to put below directories on local disk. :: #/install (Not required when using shared /install directory) @@ -263,10 +301,9 @@ To update the xCAT software in the image at a later time: :: - genimage rhels7.3-ppc64le-netboot-service - packimage rhels7.3-ppc64le-netboot-service - nodeset service osimage=rhels7.3-ppc64le-netboot-service - rsetboot service + genimage "" + packimage "" + rinstall service osimage="" Note: The Service Nodes are set up as NFS-root servers for the compute nodes. Any time changes are made to any compute image on the mgmt node it will be From 469babefab8e932a3462836669513e354b0caaf7 Mon Sep 17 00:00:00 2001 From: Casandra Qiu Date: Wed, 1 Nov 2017 13:03:20 -0400 Subject: [PATCH 5/8] bmcdiscover blathering garbage on non OpenBMC nodes --- xCAT-server/lib/xcat/plugins/bmcdiscover.pm | 2 -- 1 file changed, 2 deletions(-) diff --git a/xCAT-server/lib/xcat/plugins/bmcdiscover.pm b/xCAT-server/lib/xcat/plugins/bmcdiscover.pm index 9c07b5427..18a6fec22 100644 --- a/xCAT-server/lib/xcat/plugins/bmcdiscover.pm +++ b/xCAT-server/lib/xcat/plugins/bmcdiscover.pm @@ -692,8 +692,6 @@ sub scan_process { } } } - bmcdiscovery_openbmc(${$live_ip}[$i], $opz, $opw, $request_command) unless ($flag); - close($parent_fd); exit 0; } else { From 01bf03de7440fcc8156cf30b88e810211668b735 Mon Sep 17 00:00:00 2001 From: cxhong Date: Wed, 1 Nov 2017 22:46:04 -0400 Subject: [PATCH 6/8] switch_macmap output showed mac address on the wrong port for cumulus (#4225) switch --- perl-xCAT/xCAT/MacMap.pm | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/perl-xCAT/xCAT/MacMap.pm b/perl-xCAT/xCAT/MacMap.pm index 8b87f11a2..34f303aaa 100644 --- a/perl-xCAT/xCAT/MacMap.pm +++ b/perl-xCAT/xCAT/MacMap.pm @@ -760,6 +760,7 @@ sub refresh_switch { printf $output "$mymac|%s\n", $self->{switches}->{$switch}->{$myport}; } } + $macport="swp".$macport; push @{ $index_to_vlan{$macport} }, $macvlan; push @{ $index_to_mac{$macport} }, $mymac; } @@ -840,15 +841,15 @@ sub refresh_switch { my $port_index = $boid; my $port_name = $namemap->{ $bridgetoifmap->{$port_index} }; my $mtu = $iftomtumap->{ $bridgetoifmap->{$port_index} }; - if (defined($index_to_mac{$port_index})) { - push @{ $self->{macinfo}->{$switch}->{$port_name} }, @{ $index_to_mac{$port_index} }; + if (defined($index_to_mac{$port_name})) { + push @{ $self->{macinfo}->{$switch}->{$port_name} }, @{ $index_to_mac{$port_name} }; } else { $self->{macinfo}->{$switch}->{$port_name}->[0] = ''; } - if (defined($index_to_vlan{$port_index})) { - push @{ $self->{vlaninfo}->{$switch}->{$port_name} }, @{ $index_to_vlan{$port_index} }; + if (defined($index_to_vlan{$port_name})) { + push @{ $self->{vlaninfo}->{$switch}->{$port_name} }, @{ $index_to_vlan{$port_name} }; } else { $self->{vlaninfo}->{$switch}->{$port_name}->[0] = ''; From d3160bda34161dd8b868f47c489639ee4eeb4d92 Mon Sep 17 00:00:00 2001 From: ertaozh Date: Thu, 2 Nov 2017 02:38:22 -0400 Subject: [PATCH 7/8] modify release info for 2.13.8 --- docs/source/conf.py | 2 +- docs/source/overview/xcat2_release.rst | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index 9d1b248b3..f62bace59 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -59,7 +59,7 @@ author = u'IBM Corporation' # The short X.Y version. version = '2' # The full version, including alpha/beta/rc tags. -release = '2.13.6' +release = '2.13.8' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. diff --git a/docs/source/overview/xcat2_release.rst b/docs/source/overview/xcat2_release.rst index b3391d6b6..53c2c5c89 100644 --- a/docs/source/overview/xcat2_release.rst +++ b/docs/source/overview/xcat2_release.rst @@ -14,6 +14,16 @@ xCAT 2.13.x |xCAT |New OS |New |New Feature | |Version | |Hardware | | +=================================+===============+=============+==================================+ +|| xCAT 2.13.8 | | |- OpenBMC support: | +|| 2017/11/3 | | | | +|| | | | rflash/rpower run parallelly | +| `2.13.8 Release Notes `_ | | | | +| | | |- The document of MN HA | +| | | | | +| | | |- nodeset enhance for multi-nodes | ++---------------------------------+---------------+-------------+----------------------------------+ || xCAT 2.13.7 | | |- OpenBMC support: | || 2017/9/22 | | | | || | | | rflash for OpenBMC and PNOR | From cc5eaa15bac9e93d434d68362205a48e87ea663c Mon Sep 17 00:00:00 2001 From: Bin Xu Date: Thu, 2 Nov 2017 14:54:21 +0800 Subject: [PATCH 8/8] Fix the wrong path for xCATsn in both diskless_sn and diskfull_sn --- .../advanced/hierarchy/provision/diskful_sn.rst | 10 +++++----- .../advanced/hierarchy/provision/diskless_sn.rst | 11 +++++++++-- 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/docs/source/advanced/hierarchy/provision/diskful_sn.rst b/docs/source/advanced/hierarchy/provision/diskful_sn.rst index 972a81712..e882a50b1 100644 --- a/docs/source/advanced/hierarchy/provision/diskful_sn.rst +++ b/docs/source/advanced/hierarchy/provision/diskful_sn.rst @@ -33,22 +33,22 @@ Configure ``otherpkgdir`` and ``otherpkglist`` for service node osimage * Verify the following entries are included in the package file specified by the ``otherpkglist`` attribute of the service node osimage. :: - xcat/xcat-dep///xCATsn + xcat/xcat-core/xCATsn xcat/xcat-dep///conserver-xcat xcat/xcat-dep///perl-Net-Telnet xcat/xcat-dep///perl-Expect For example, for the osimage *rhels7-x86_64-install-service* :: - [root@fs4 ~]# lsdef -t osimage rhels7-x86_64-install-service -i otherpkglist - Object name: rhels7-x86_64-install-service + lsdef -t osimage rhels7-x86_64-install-service -i otherpkglist + Object name: rhels7-x86_64-install-service otherpkglist=/opt/xcat/share/xcat/install/rh/service.rhels7.x86_64.otherpkgs.pkglist - [root@fs4 ~]# cat /opt/xcat/share/xcat/install/rh/service.rhels7.x86_64.otherpkgs.pkglist + + cat /opt/xcat/share/xcat/install/rh/service.rhels7.x86_64.otherpkgs.pkglist xcat/xcat-core/xCATsn xcat/xcat-dep/rh7/x86_64/conserver-xcat xcat/xcat-dep/rh7/x86_64/perl-Net-Telnet xcat/xcat-dep/rh7/x86_64/perl-Expect - [root@fs4 ~]# **Note:** you will be installing the xCAT Service Node rpm xCATsn meta-package on the Service Node, not the xCAT Management Node meta-package. Do not install both. diff --git a/docs/source/advanced/hierarchy/provision/diskless_sn.rst b/docs/source/advanced/hierarchy/provision/diskless_sn.rst index 173c45c73..95abb79e6 100644 --- a/docs/source/advanced/hierarchy/provision/diskless_sn.rst +++ b/docs/source/advanced/hierarchy/provision/diskless_sn.rst @@ -83,12 +83,19 @@ When you run ``copycds``, xCAT will only create a Service Node stateful osimage :: - xcat/xcat-dep///xCATsn + xcat/xcat-core/xCATsn xcat/xcat-dep///conserver-xcat xcat/xcat-dep///perl-Net-Telnet xcat/xcat-dep///perl-Expect - These are required to install the xCAT Service Node function into your image. + For example, for the osimage *rhels7.3-ppc64le-netboot-service*: :: + + xcat/xcat-core/xCATsn + xcat/xcat-dep/rh7/ppc64le/conserver-xcat + xcat/xcat-dep/rh7/ppc64le/perl-Net-Telnet + xcat/xcat-dep/rh7/ppc64le/perl-Expect + + **Note:** you will be installing the xCAT Service Node rpm xCATsn meta-package on the Service Node, not the xCAT Management Node meta-package. Do not install both. 2, Create the postinstall script.