test(xcat-test): the fast tests of a change set cannot be run as one suite

The service node work landed seven tests, one per fix, and running them meant
knowing which are Perl and which are shell and typing two commands with seven
paths. Delta debugging the change set runs that suite once per candidate, so
the alternative was the end-to-end suite at 45 to 90 minutes a round.

quick.sh runs a named set, sending .t files to prove and .bats files to bats,
and with no argument runs the whole fast suite, which is what CI runs. A named
file that the tree does not have is an error rather than a skip: reporting PASS
for a tree containing none of the tests is how a minimization drops the commit
that adds one.

quick-servicenode.txt names this change set's seven, with what each covers and
what none of them covers.

Measured: 2.6 seconds for the seven against 45 to 90 minutes for one end-to-end
cell. The list passes on the full branch and fails on the base with these same
test files, which is what makes it usable as an oracle.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>

xCAT-test/quick-servicenode.txt

@@ -0,0 +1,32 @@
+# The fast oracle of the service node change set: one test per fix, each written with its fix.
+#
+# Run it with:  xCAT-test/quick.sh -f xCAT-test/quick-servicenode.txt
+#
+# These are the tests that failed before their fix landed. Together they stand in for the
+# end-to-end service node suite, which provisions a management node, a service node and a
+# compute node on two families and takes 45 to 90 minutes a run.
+#
+# What each one covers:
+#   check_provisioning_source   the compute node was served by the service node, not the MN
+#   makenamed_forwarders        a Debian service node does not forward DNS to its own stub
+#   otherpkgs_apt_source_trusted  the staged apt source apt will accept
+#   stage_sn_apt_repo           the Debian repository index, where createrepo does not exist
+#   servicenode_builder_repo    the builder repository per EL release: crb, powertools, codeready
+#   svrutils_nfs_export_line    a service node that mounts /install still exports it
+#   ubuntu_service_subiquity_template  the service profile resolves a Subiquity template
+#
+# A SKIP is not a PASS. otherpkgs_apt_source_trusted skips its two resolving assertions where
+# apt-get is absent, so on an EL builder that file proves less than it does on a Debian one. Run
+# this list on the builder of the family the candidate touches, or read the skips.
+#
+# NOT covered, and known: the xdsh PATH fix, and the named working directory and reload fixes.
+# A candidate that removes those passes this list and fails the end-to-end suite. Fill these in
+# before trusting the list for a minimization.
+
+xCAT-test/bats/check_provisioning_source.bats
+xCAT-test/bats/makenamed_forwarders.bats
+xCAT-test/bats/otherpkgs_apt_source_trusted.bats
+xCAT-test/bats/stage_sn_apt_repo.bats
+xCAT-test/unit/servicenode_builder_repo.t
+xCAT-test/unit/svrutils_nfs_export_line.t
+xCAT-test/unit/ubuntu_service_subiquity_template.t

xCAT-test/quick.sh

@@ -0,0 +1,79 @@
+#!/bin/bash
+#
+# Run a fast subset of the unit tests: the Perl .t files with prove and the shell .bats files
+# with bats, in one command, from the source tree. No build, no installed xCAT, no cluster.
+#
+# WHY THIS EXISTS. Delta debugging a change set runs the suite once per candidate. The
+# end-to-end suite provisions machines and costs 45 to 90 minutes a run, so the rounds run on a
+# fast oracle instead, and the slow suite runs once on the set that settles. This is that fast
+# oracle, and it is useful on its own: it is the same two commands CI runs, without the wait.
+#
+#   xCAT-test/quick.sh                      every unit test and every bats file
+#   xCAT-test/quick.sh path/a.t path/b.bats only those, in the order given
+#   xCAT-test/quick.sh -f list.txt          the files named in list.txt, one per line
+#
+# Exit status is 0 only when every file passed. A missing file is an error, not a skip: a fast
+# oracle that silently runs nothing reports PASS for a tree that contains none of the tests.
+
+set -u -o pipefail
+
+usage() { sed -n '3,20p' "$0" | sed 's/^# \{0,1\}//'; exit "${1:-0}"; }
+
+root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
+files=()
+
+while [ $# -gt 0 ]; do
+    case "$1" in
+        -h|--help) usage 0 ;;
+        -f|--from) shift; [ -r "${1:-}" ] || { echo "quick: cannot read list '${1:-}'" >&2; exit 2; }
+                   while read -r line; do
+                       case "$line" in ''|\#*) continue ;; esac
+                       files+=("$line")
+                   done < "$1" ;;
+        -*) echo "quick: unknown option '$1'" >&2; usage 2 ;;
+        *)  files+=("$1") ;;
+    esac
+    shift
+done
+
+cd "$root" || exit 2
+
+if [ "${#files[@]}" -eq 0 ]; then
+    echo "[quick] whole fast suite: prove -r xCAT-test/unit, then bats -r xCAT-test/bats"
+    rc=0
+    prove -r xCAT-test/unit || rc=1
+    bats  -r xCAT-test/bats || rc=1
+    exit $rc
+fi
+
+# A named file that is absent means the caller asked for a test this tree does not have. Say so
+# and fail: reporting PASS here is how a minimization drops the commit that adds a test.
+missing=()
+for f in "${files[@]}"; do [ -e "$f" ] || missing+=("$f"); done
+if [ "${#missing[@]}" -gt 0 ]; then
+    printf '[quick] MISSING from this tree, refusing to report a result:\n' >&2
+    printf '  %s\n' "${missing[@]}" >&2
+    exit 2
+fi
+
+perl_tests=(); bats_tests=()
+for f in "${files[@]}"; do
+    case "$f" in
+        *.t)    perl_tests+=("$f") ;;
+        *.bats) bats_tests+=("$f") ;;
+        *)      echo "quick: '$f' is neither a .t nor a .bats" >&2; exit 2 ;;
+    esac
+done
+
+rc=0
+if [ "${#perl_tests[@]}" -gt 0 ]; then
+    echo "[quick] prove ${#perl_tests[@]} Perl test(s)"
+    prove "${perl_tests[@]}" || rc=1
+fi
+if [ "${#bats_tests[@]}" -gt 0 ]; then
+    echo "[quick] bats ${#bats_tests[@]} shell test(s)"
+    bats "${bats_tests[@]}" || rc=1
+fi
+
+[ $rc -eq 0 ] && echo "[quick] PASS (${#files[@]} file(s))" || echo "[quick] FAIL"
+exit $rc