diff --git a/docs/source/guides/admin-guides/references/man1/xcatperftest.1.rst b/docs/source/guides/admin-guides/references/man1/xcatperftest.1.rst new file mode 100644 index 000000000..4911358cd --- /dev/null +++ b/docs/source/guides/admin-guides/references/man1/xcatperftest.1.rst @@ -0,0 +1,213 @@ + +############## +xcatperftest.1 +############## + +.. highlight:: perl + + +**** +NAME +**** + + +\ **xcatperftest**\ - Run xCAT command performance baseline testing on fake nodes. + + +******** +SYNOPSIS +******** + + +\ **xcatperftest**\ [\ **-?|-h**\ ] + +[\ **PERF_DRYRUN**\ =y] [\ **PERF_NOCREATE**\ =y] \ **xcatperftest**\ [\ *command-list-file*\ ] + + +*********** +DESCRIPTION +*********** + + +The xcatperftest command runs commandes defined in a command list file and get their execution response time baseline for performance purpose. +The xcatperftest command is part of the xCAT package xCAT-test, and you can run it standalone or leverage it to build up your automation test cases. + +Any commands could be defined in the command list file, however, it is recommended that the one-time initial configuration are well prepared prior to run xcatperftest command. +For example, the network object, osdistor and osimage image objects. + +Follow the below steps to run xcatperftest command: + +1, Install xCAT-test on a xCAT management nodes. + +2, Prepare a command list in which the commands are what you want to messure. + +3, Prepare the initial configuration based on the command list to make sure all commands could be executed in techinal. + +4, Run xcatperftest with the total fake nodes number and the above command list file. + +Node: It is suggested to run the command in background as it normally takes long time to finish all the performanc testing with large amount of fake nodes. + + +******* +OPTIONS +******* + + + +\ **-?|-h**\ + + Display usage message. + + + + + + Specifies the command list file with full-path. xCAT supports an example command file: /opt/xcat/share/xcat/tools/autotest/perfcmds.lst + + + + + + Total number of fake nodes will be defined during the testing. + + + + +************ +RETURN VALUE +************ + + +0 The command completed successfully. + +1 An error has occurred. + + +***************** +COMMAND LIST FILE +***************** + + +The command list file is in flat text format, the testing framework will parse the file line by line, here is an example of the commannd list file: + + +.. code-block:: perl + + #SERIES# 1,50,100,250,500,1000,2500,5000 + mkdef -z -f < #STANZ# + lsdef #NODES# + makehosts #NODES# + makedns -n #NODES# + makedhcp #NODES# + makeknownhosts #NODES# + nodech #NODES# groups,=group1 + nodels #NODES# noderes + nodeset #NODES# osimage=rhels7.3-GA-ppc64le-install-compute + chdef -t node -o #NODES# postscripts="fake" profile="install" netboot="grub2" + rmdef -t node #PERFGRP# + mkdef -z < #STANZ# + noderm #PERFGRP# + + +\ **Note**\ : Each line defines one command, and the commands dependency should be handled by the line order. +If you define a node range series line (started with #SERIES#) in this file, xcatperftest will run the command for each node range defined in series line. + +\ **#SERIES#**\ To define a node range series, and the series should be an comma split incremental number sequence. + +\ **#STANZ#**\ It will be replaced with real stanz file path when this command line runs. + +\ **#NODES#**\ It will be replaced with real node range defined in #SERIES# line when this command line runs. If no series line, the node group will be used. + +\ **#PERFGRP#**\ It will be replaced with node group when this command line runs. + + +******************** +ENVIRONMENT VARIABLE +******************** + + +The xcatperftest command supports be customized by some environment variables. + +\ **FAKE_NODE_PREFIX**\ + +Optional, the prefix of the fake compute node name. By default, the value is 'fake' + +\ **FAKE_NODE_GROUP**\ + +# Optional, the group name of all the fake compute nodes. By default, the value is 'perftest' + +\ **FAKE_NETWORK_PRO**\ + +Mandatory, the Provision network for all the fake compute nodes. By default, the value is '192.168'. +It must be a string like 'A.B', and be matched with \`tabdump networks\` + +\ **FAKE_NETWORK_BMC**\ + +Mandatory, the BMC network for all the fake compute nodes. By default, the value is '192.168'. Note: It could not be the same subnet as 'FAKE_NETWORK_PRO' +It must be a string like 'A.B' and no need to be defined in 'networks' table. + +\ **PERF_NODETEMPL**\ + +Optional, The node template name used for generating fake nodes. By default, it will be auto-detected according to the current arch. + +\ **PERF_DRYRUN**\ + +Optional, Indicate no real commands will be executed if the environment variable is set. + +\ **PERF_NOCREATE**\ + +Optional, Indicate no new fake nodes will be created if the environment variable is set. + + +******** +EXAMPLES +******** + + + +1. + + To run the performance testing for the commands defined in /tmp/cmd.lst on 5000 fake nodes: + + + .. code-block:: perl + + xcatperftest 5000 /tmp/cmd.lst + + + + +2. + + To generate an xCAT node object stanz file for 10000 nodes in subnet 10.100.0.0: + + + .. code-block:: perl + + FAKE_NETWORK_PRO=10.100 FAKE_NETWORK_BMC=10.200 xcatperftest 10000 + + + + +3. + + To run the performance testing for the commands defined in /opt/xcat/share/xcat/tools/autotest/perfcmds.lst on 5000 existing fake nodes: + + + .. code-block:: perl + + PERF_NOCREATE=y xcatperftest 5000 /opt/xcat/share/xcat/tools/autotest/perfcmds.lst + + + + + +***** +FILES +***** + + +/opt/xcat/bin/xcatperftest + +/opt/xcat/share/xcat/tools/autotest/perfcmds.lst + diff --git a/xCAT-test/autotest/perfcmds.lst b/xCAT-test/autotest/perfcmds.lst new file mode 100644 index 000000000..e1cf7eb3c --- /dev/null +++ b/xCAT-test/autotest/perfcmds.lst @@ -0,0 +1,13 @@ +#SERIES# 1,50,100,250,500,1000,2500,5000,10000 +lsdef #NODES# +makehosts #NODES# +makedns -n #NODES# +makedhcp #NODES# +makeknownhosts #NODES# +nodech #NODES# groups,=group1 +nodels #NODES# noderes +nodeset #NODES# osimage=rhels7.3-GA-ppc64le-install-compute +chdef -t node -o #NODES# postscripts="fake" profile="install" netboot="grub2" +rmdef -t node #PERFGRP# +mkdef -z < #STANZ# +noderm #PERFGRP# diff --git a/xCAT-test/bin/xcatperftest b/xCAT-test/bin/xcatperftest new file mode 100755 index 000000000..e68e6b8f4 --- /dev/null +++ b/xCAT-test/bin/xcatperftest @@ -0,0 +1,294 @@ +#!/bin/bash +# IBM(c) 2017 EPL license http://www.eclipse.org/legal/epl-v10.html +#(C)IBM Corp +# +################################################################### +# +# Description: +# This script is used for performance testing purpose. It could +# generate n*250 fake nodes based on a predefined template (/opt/xcat/share/xcat/templates/objects/node/), +# and then run the performance testing on a batch of xCAT commands. +# +# Note: It is availabe for the commands which require management node only. +# +# Syntax: +# $prog [command-list-file] +# +################################################################### + +if [ -z $LC_ALL ]; then + export LC_ALL=C +fi + +# Used for number parameter validation +isNumber() +{ + expr $1 + 0 &>/dev/null +} + +# Give a simple usage +if [ -z "$1" ] || [ "$1" = "-h" ] || [ "$1" = "--help" ]; then + echo "[PERF_DRYRUN=y] [PERF_NOCREATE=y] $0 [command-list-file]" + exit +else + isNumber $1 + if [ ! $? -eq 0 ]; then + echo "You must input an numeric string for total nodes number." + exit -1 + fi +fi + +# Source xCAT profile to make sure the xCAT commands could be executed without absolute path. +if [ -z $XCATROOT ]; then + if [ -r /etc/profile.d/xcat.sh ]; then + . /etc/profile.d/xcat.sh + fi +fi + +# Mandatory, to specify the number of total fake nodes which will be created for testing +FAKE_NODE_TOTAL=$1 + +# Optional, the prefix of the fake compute node name. +# By default, it is 'fake' but it could be changed when you set environment variable `FAKE_NODE_PREFIX` +if [ -z $FAKE_NODE_PREFIX ]; then + FAKE_NODE_PREFIX='fake' +fi + +# Optional, the group name of all the fake compute nodes. +# By default, it is 'perftest' but it could be changed when you set environment variable `FAKE_NODE_GROUP` +if [ -z $FAKE_NODE_GROUP ]; then + FAKE_NODE_GROUP='perftest' +fi + +# Mandatory, the Provision network for all the fake compute nodes. It must be a string like 'A.B', and be matched with `tabdump networks` +# By default, it is '192.168' but it could be changed when you set environment variable `FAKE_NETWORK_PRO` +if [ -z $FAKE_NETWORK_PRO ]; then + FAKE_NETWORK_PRO='192.168' +fi + +# Mandatory, the BMC network for all the fake compute nodes. It must be a string like 'A.B' and no need to be defined in 'networks' table. +# By default, it is '192.169' but it could be changed when you set environment variable `FAKE_NETWORK_BMC` +# Note: it could not be the same subnet as 'FAKE_NETWORK_PRO' +if [ -z $FAKE_NETWORK_BMC ]; then + FAKE_NETWORK_BMC='192.169' +fi + +# Optional, The node template name used for generating fake nodes. +# By default, it is '-template' but it could be changed when you set environment variable `FAKE_NODE_GROUP` +if [ -z $PERF_NODETEMPL ]; then + PERF_NODETEMPL="`arch`-template" +fi + +# IP address assinged to node will be in [1-250] +NODE_PER_ROW=250 + +PERFORMANCE_DIR=/tmp/xcat-performance +PERFORMANCE_NODE_TMPL=$PERFORMANCE_DIR/node.tmpl +PERFORMANCE_REPORT=$PERFORMANCE_DIR/report-$FAKE_NODE_TOTAL.$$ +PERFORMANCE_STANZ=$PERFORMANCE_DIR/stanz-$FAKE_NODE_TOTAL.$$ + +# If the command list file is not specified, the tool will only create the stanz file for fake nodes. +# If it is specified but not exists, the tool will exit with error. +if [ ! -z $2 ]; then + if [ -f $2 ]; then + RUN_CMD_LIST=$2 + else + echo "ERROR: The command list file you specified does not exist." + exit -1 + fi +fi + + +# Get a random MAC address +genMAC() +{ + printf '00:60:2F:%02X:%02X:%02X' $[RANDOM%256] $[RANDOM%256] $[RANDOM%256] +} + +# Generate stanz file for all fake nodes +genStanz() +{ + if [ ! -z $PERF_DRYRUN ]; then + echo $1, $2, $3, $4 + else + echo -n . + fi + + sed -e '/Object name:/c \'"$1"':\n objtype=node' \ + -e '/ip=/c \ ip='"$3"'' \ + -e '/mac=/c \ mac='"$2"'' \ + -e '/bmc=/c \ bmc='"$4"'' \ + -e '/bmcusername=/c \ bmcusername=fake' \ + -e '/bmcpassword=/c \ bmcpassword=fake' \ + -e '/groups=/c \ groups=all,'"$FAKE_NODE_GROUP"'' \ + -e '/postscripts=/c \ postscripts=mypostboot' \ + -e '/postbootscripts=/c \ postbootscripts=mypostboot' \ + $PERFORMANCE_NODE_TMPL >> $PERFORMANCE_STANZ +} + +# Create a fake xCAT node definition +fakeNode() +{ + # TODO: support regular expression for IP + genStanz $FAKE_NODE_PREFIX$1 $(genMAC) $FAKE_NETWORK_PRO.$2.$3 $FAKE_NETWORK_BMC.$2.$3 + #mkdef -f -t node $FAKE_NODE_PREFIX$1 --template $PERF_NODETEMPL ip=$FAKE_NETWORK_PRO.$2.$3 mac=$(genMAC) \ + # bmc=$FAKE_NETWORK_BMC.$2.$3 bmcpassword=fake bmcusername=fake groups=all,performance > /dev/null 2>&1 +} + +# Create batch fake nodes stanz file for testing +bootstrap() +{ + declare -i count=0 + [ $rack = 0 ] && rack=1 + for i in $(seq 0 $(expr $rack - 1)) + do + for j in $(seq 1 $NODE_PER_ROW) + do + count+=1 + fakeNode $count $i $j + [ $(($count % $1)) = 0 ] && echo + [ "x$count" == "x$FAKE_NODE_TOTAL" ] && break + done + done +} + +# Get current time +getTime() +{ + date +%s -d "$1" +} + +# Executing the testing on specific commands defined in command list file +# All MACROs defined in command list file will be replaced with the real value +runTest() +{ + + cmd=$1 + if [[ $cmd =~ '#STANZ#' ]]; then + #mkdef -z + execCmd "${cmd/\#STANZ\#/$PERFORMANCE_STANZ}" "$FAKE_NODE_TOTAL" + + elif [[ $cmd =~ '#NODES#' ]]; then + #noderange operation + if [ -z "$2" ]; then + # No SERIES defined, run command on the whole group + execCmd "${cmd/\#NODES\#/$FAKE_NODE_GROUP}" "$FAKE_NODE_TOTAL" + else + # run the command for each number in SERIES + for num in $2 + do + isNumber $num || continue + if [[ $num -le $FAKE_NODE_TOTAL ]]; then + cmd=$1 + execCmd "${cmd/\#NODES\#/$FAKE_NODE_PREFIX[1-$num]}" "$num" + fi + done + fi + elif [[ $cmd =~ '#PERFGRP#' ]]; then + execCmd "${cmd/\#PERFGRP\#/$FAKE_NODE_GROUP}" "$FAKE_NODE_TOTAL" + else + execCmd "$cmd" "N/A" + fi + +} + +# Output performance result for each command. +printResult() +{ + cmd=`echo "$1" | awk '{print $1}'` + result=$([[ $4 = 0 ]] && echo "SUCESS" || echo "FAIL") + # TOTAL, CMD, NODERANGE, TIME, SUCESS, FULL COMMAND + echo "$FAKE_NODE_TOTAL", "$cmd", "$2", "$3", $result, \"$1\" >> $PERFORMANCE_REPORT +} + +# Executing each command and print the result to report file +execCmd() +{ + echo "Testing for [ $1 ] ..." + if [ ! -z $PERF_DRYRUN ]; then + return + fi + + starttime=`date +'%Y-%m-%d %H:%M:%S'` + start=$(getTime "$starttime") + eval "$1" > /dev/null 2>&1 + retval=$? + endtime=`date +'%Y-%m-%d %H:%M:%S'` + end=$(getTime "$endtime") + + printResult "$1" "$2" "$(($end-$start))" "$retval" +} + +################################################# +# Main Loop of the performance baseline testing # +################################################# +lsxcatd -a +if [ 0 != $? ]; then + echo "ERROR: xCAT daemon is not running" + #exit 99 +fi + +mkdir -p /tmp/xcat-performance + +lsdef -t node --template $PERF_NODETEMPL > $PERFORMANCE_NODE_TMPL 2>/dev/null +if [ 0 != $? ]; then + echo "ERROR: Cannot find the default template for `arch`, make sure it exists and rerun the script." + exit 99 +fi + +rack=$(expr $FAKE_NODE_TOTAL / $NODE_PER_ROW) + +echo "==================================================" +# Starting to add fake nodes +starttime=`date +'%Y-%m-%d %H:%M:%S'` +start=$(getTime "$starttime") +bootstrap 50 +endtime=`date +'%Y-%m-%d %H:%M:%S'` +end=$(getTime "$endtime") +echo +echo "==================================================" + +#echo "It takes $(($end-$start)) seconds to create $FAKE_NODE_TOTAL nodes" +#echo "$FAKE_NODE_TOTAL", "$(($end-$start))", "mkdef" >> $PERFORMANCE_REPORT + +if [ -z $RUN_CMD_LIST ]; then + echo "Done. Check the stanz file in $PERFORMANCE_STANZ" + exit 0 +fi + +echo "Continue the performance testing for commands in $RUN_CMD_LIST " +echo "==================================================" + +# Initial Populate the fake nodes into DB +if [ -z $PERF_NOCREATE ]; then + execCmd "mkdef -z -f < $PERFORMANCE_STANZ" "$FAKE_NODE_TOTAL" +fi + +series=`grep '^#SERIES#' $RUN_CMD_LIST | awk '{print $2}'` +if [ ! -z $series ]; then + series=${series//,/ } +fi +#echo $series + +cmdlist=`cat $RUN_CMD_LIST` +IFS_BAK=$IFS +IFS=$'\n' +for line in $cmdlist +do + [ "x${line:0:1}" = "x#" ] && continue + + # begin to run the command + IFS=$IFS_BAK + runTest "$line" "$series" + IFS=$'\n' +done +IFS=$IFS_BAK +IFS_BAK= + +rm -f $PERFORMANCE_NODE_TMPL +rm -f $PERFORMANCE_STANZ + +if [ -z $PERF_DRYRUN ]; then + echo + echo "Done. Check the performance result in $PERFORMANCE_REPORT" +fi diff --git a/xCAT-test/debian/install b/xCAT-test/debian/install index 12aefb56a..5a16deaab 100644 --- a/xCAT-test/debian/install +++ b/xCAT-test/debian/install @@ -1,4 +1,5 @@ xcattest opt/xcat/bin +bin/xcatperftest opt/xcat/bin share/man/man1/* opt/xcat/share/man/man1 share/doc/man1/* opt/xcat/share/doc/man1 -autotest opt/xcat/share/xcat/tools \ No newline at end of file +autotest opt/xcat/share/xcat/tools diff --git a/xCAT-test/pods/man1/xcatperftest.1.pod b/xCAT-test/pods/man1/xcatperftest.1.pod new file mode 100644 index 000000000..2f1ef8663 --- /dev/null +++ b/xCAT-test/pods/man1/xcatperftest.1.pod @@ -0,0 +1,151 @@ +=head1 NAME + +B - Run xCAT command performance baseline testing on fake nodes. + +=head1 SYNOPSIS + +B [B<-?|-h>] + +[B=y] [B=y] B [I] + + +=head1 DESCRIPTION + +The xcatperftest command runs commandes defined in a command list file and get their execution response time baseline for performance purpose. +The xcatperftest command is part of the xCAT package xCAT-test, and you can run it standalone or leverage it to build up your automation test cases. + +Any commands could be defined in the command list file, however, it is recommended that the one-time initial configuration are well prepared prior to run xcatperftest command. +For example, the network object, osdistor and osimage image objects. + +Follow the below steps to run xcatperftest command: + +1, Install xCAT-test on a xCAT management nodes. + +2, Prepare a command list in which the commands are what you want to messure. + +3, Prepare the initial configuration based on the command list to make sure all commands could be executed in techinal. + +4, Run xcatperftest with the total fake nodes number and the above command list file. + +Node: It is suggested to run the command in background as it normally takes long time to finish all the performanc testing with large amount of fake nodes. + +=head1 OPTIONS + +=over 10 + +=item B<-?|-h> + +Display usage message. + +=item + +Specifies the command list file with full-path. xCAT supports an example command file: /opt/xcat/share/xcat/tools/autotest/perfcmds.lst + +=item + +Total number of fake nodes will be defined during the testing. + +=back + +=head1 RETURN VALUE + +0 The command completed successfully. + +1 An error has occurred. + + +=head1 COMMAND LIST FILE + +The command list file is in flat text format, the testing framework will parse the file line by line, here is an example of the commannd list file: + + #SERIES# 1,50,100,250,500,1000,2500,5000 + mkdef -z -f < #STANZ# + lsdef #NODES# + makehosts #NODES# + makedns -n #NODES# + makedhcp #NODES# + makeknownhosts #NODES# + nodech #NODES# groups,=group1 + nodels #NODES# noderes + nodeset #NODES# osimage=rhels7.3-GA-ppc64le-install-compute + chdef -t node -o #NODES# postscripts="fake" profile="install" netboot="grub2" + rmdef -t node #PERFGRP# + mkdef -z < #STANZ# + noderm #PERFGRP# + + +B: Each line defines one command, and the commands dependency should be handled by the line order. +If you define a node range series line (started with #SERIES#) in this file, xcatperftest will run the command for each node range defined in series line. + +B<#SERIES#> To define a node range series, and the series should be an comma split incremental number sequence. + +B<#STANZ#> It will be replaced with real stanz file path when this command line runs. + +B<#NODES#> It will be replaced with real node range defined in #SERIES# line when this command line runs. If no series line, the node group will be used. + +B<#PERFGRP#> It will be replaced with node group when this command line runs. + +=head1 ENVIRONMENT VARIABLE + +The xcatperftest command supports be customized by some environment variables. + + +B + +Optional, the prefix of the fake compute node name. By default, the value is 'fake' + +B + +# Optional, the group name of all the fake compute nodes. By default, the value is 'perftest' + +B + +Mandatory, the Provision network for all the fake compute nodes. By default, the value is '192.168'. +It must be a string like 'A.B', and be matched with `tabdump networks` + +B + +Mandatory, the BMC network for all the fake compute nodes. By default, the value is '192.168'. Note: It could not be the same subnet as 'FAKE_NETWORK_PRO' +It must be a string like 'A.B' and no need to be defined in 'networks' table. + +B + +Optional, The node template name used for generating fake nodes. By default, it will be auto-detected according to the current arch. + +B + +Optional, Indicate no real commands will be executed if the environment variable is set. + +B + +Optional, Indicate no new fake nodes will be created if the environment variable is set. + +=head1 EXAMPLES + +=over 4 + +=item 1. + +To run the performance testing for the commands defined in /tmp/cmd.lst on 5000 fake nodes: + + xcatperftest 5000 /tmp/cmd.lst + +=item 2. + +To generate an xCAT node object stanz file for 10000 nodes in subnet 10.100.0.0: + + FAKE_NETWORK_PRO=10.100 FAKE_NETWORK_BMC=10.200 xcatperftest 10000 + +=item 3. + +To run the performance testing for the commands defined in /opt/xcat/share/xcat/tools/autotest/perfcmds.lst on 5000 existing fake nodes: + + PERF_NOCREATE=y xcatperftest 5000 /opt/xcat/share/xcat/tools/autotest/perfcmds.lst + +=back + +=head1 FILES + +/opt/xcat/bin/xcatperftest + +/opt/xcat/share/xcat/tools/autotest/perfcmds.lst diff --git a/xCAT-test/xCAT-test.spec b/xCAT-test/xCAT-test.spec index 6906896da..0625e5de9 100644 --- a/xCAT-test/xCAT-test.spec +++ b/xCAT-test/xCAT-test.spec @@ -42,6 +42,7 @@ mkdir -p $RPM_BUILD_ROOT/%{prefix}/share/xcat/tools/autotest mkdir -p $RPM_BUILD_ROOT/%{prefix}/share/man/man1 mkdir -p $RPM_BUILD_ROOT/%{prefix}/share/doc/man1 +cp bin/* $RPM_BUILD_ROOT/%{prefix}/bin cp xcattest $RPM_BUILD_ROOT/%{prefix}/bin chmod 755 $RPM_BUILD_ROOT/%{prefix}/bin/*