The step that fills lib/firmware in the Ubuntu netboot initrd asked modinfo
about $rootimg_dir/$module and looked a firmware name up under lib/firmware.
The copy step beside it takes a module from $customdir or $pathtofiles first,
and the kernel looks a firmware name up under updates/<kernel>, updates/,
<kernel>/ and lib/firmware. So a custom driver reached the initrd with no
firmware, even when the root image carried it, and a firmware override was left
out of the initrd altogether.
initrd_firmware_files now takes the module directories the copy step searches
and the kernel release. It resolves each module in that order before asking
modinfo, and keeps every firmware file that exists in the four directories the
kernel searches, so the override still wins on the node.
ubuntu_genimage_initrd_firmware.t covers both: its two new cases are red on the
commit before this one.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
ubuntu_genimage_initrd_firmware.t drove the firmware step over a root image that
holds every module and every firmware file. It covered neither of the two places
genimage reads from beside the root image.
The test now puts a driver in the custom directory, with its firmware in the
root image, and a firmware override under lib/firmware/updates/<kernel>. Both
are red: the step asks modinfo about the module under the root image, where a
custom driver is not, and it looks for a firmware name under lib/firmware only,
where an override is not.
The five assertions that were there stay green.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
ubuntu_genimage_initrd_firmware.t called plan skip_all when xCAT-server/share/xcat/netboot/ubuntu/genimage was absent, so a checkout that lost
the file reported 0 tests and exit 0. A test that cannot fail measures nothing.
Die instead, which is what makentp_ntp_deps.t already does for setupntp.
With xCAT-server/share/xcat/netboot/ubuntu/genimage moved aside the file now exits 2 and prints "genimage not found at <path>";
before this change it exited 0 and printed "1..0 # SKIP genimage not found at <path>". With the file
present the test passes either way.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The ubuntu-26.04-ppc64el diskless node never starts its kernel. SLOF reports
W3411 and E3406 and falls through to disk. The node fetches the kernel and the
initrd in under a minute and fails six minutes later, with no network activity in
between: grub2 has the payload and cannot start it.
genimage copies the whole lib/firmware tree of the root image into the initrd.
That tree is 666 MB on Ubuntu 26.04, which takes the initrd to 719 MB. A 687 MB
initrd boots the same kernel on the same node; a 719 MB one does not.
The firmware copy now takes only the firmware that the drivers in the initrd ask
for, which modinfo reports for each module. The root image keeps its whole tree,
so the node that boots is unchanged.
ubuntu_genimage_initrd_firmware.t drives the firmware step over a root image whose
firmware tree holds files no driver asks for. It fails on the previous commit.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The Ubuntu genimage copies every file under the root image lib/firmware into the
netboot initrd. On Ubuntu 26.04 that tree is 666 MB, and grub2 on a pseries node
cannot load the initrd it produces.
The test runs the firmware step of genimage over a root image whose firmware tree
holds files no driver in the initrd asks for, and reads the initrd that comes out.
It fails on the present code, which copies the tree whole.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Inside the private network namespace the fake tcpdump records its parent,
its output file and the TERM it receives, ps records any use, and a second
fake tcpdump fails at once. Both copies must start tcpdump directly, write
under TMPDIR, stop it by pid, leave no file behind, and exit 1 when the
capture fails. The block skips where no namespace is available.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Both copies wrote the capture to /tmp/dhcpdumpfile.log, so a file left by
another user or an earlier root run blocked the probe and two runs
overwrote each other. tcpdump ran behind a shell, so the script killed it
by searching the process table for any tcpdump on the interface, and a
tcpdump that failed to start left an empty file that read as zero DHCP
servers. An interrupt left the capture running.
The capture file is now a private temporary file removed on every exit.
The child execs tcpdump itself, so the script stops and reaps exactly that
pid, and a tcpdump that ended before the window did fails the run. A
failed send and an INT or TERM stop the capture and exit 1 as well.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The ospkgs apt block runs with a shadowed apt-get that fails only for the
cuda install, and RETURNVAL must carry that status. Against the previous
script the test fails with RETURNVAL=0.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Both cuda install blocks read the package manager status after they had
restored and exported ARCH, so R was always 0 and a failed cuda install
left the node reporting success. The status is now read directly after
the install on the apt path and on the yum and dnf path.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
xcat_apt_get is sourced from xcatpkgutils.sh and called with a shadowed
apt-get that records its environment and arguments. The apt block of
ospkgs and the two apt commands of otherpkgs are extracted from the
scripts and executed the same way, since neither script can be sourced.
Against the previous scripts 6 of the 7 tests fail: the helper does not
exist, and the recorded calls carry --force-yes.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
ospkgs and otherpkgs passed --force-yes to every apt-get upgrade and
install. apt has printed a deprecation warning for it since 1.1, and the
flag also allowed downgrades, changes to held packages and the removal of
essential packages, none of which an unattended package update should do.
Three of the ospkgs installs relied on it alone for the unsigned xCAT
repositories and ran without DEBIAN_FRONTEND=noninteractive, unlike the
upgrade.
The 8 call sites now go through xcat_apt_get in xcatpkgutils.sh, which
runs apt-get with -y and --allow-unauthenticated under
DEBIAN_FRONTEND=noninteractive. Each call site keeps its own quiet, Dpkg
and recommends options. A held package or an explicit older version now
fails the install instead of being forced, as on the rpm side.
--allow-unauthenticated is accepted by every apt release xCAT provisions.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The control files are parsed the way dpkg reads them, one stanza per binary
package, and the declarations of perl-xcat, xcat-client, xcat-server, xcat
and xcatsn are held to the modules and tools their files use, with the
ipmitool-xcat floor. 25 of 26 assertions fail against the previous
control files.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The deb metapackages and xcat-client only recommended nmap and
ipmitool-xcat, while xCAT.spec, xCATsn.spec and xCAT-client.spec require
them. An install with --no-install-recommends therefore left bmcdiscover
without a scan method and out-of-band management without the tool ipmi.pm
and bmcdiscover.pm hard-code. Both move to Depends.
The ipmitool-xcat floor rises from 1.8.17-1 to 1.8.18-4, the floor of
xCAT.spec. Builds before 1.8.18-3 lack the CVE-2020-5208 patch.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
xcat-server ships xcatws.cgi and restapi.pl with the endpoint enabled in
apache, and both load CGI at top level. Nothing declared the module, so on
a management node where nothing else pulled it every /xcatws request
answered 500. The rpm package requires perl(CGI).
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
zxcatexport.pl, zxcatimport.pl and zxcatCopyCloneList.pl load
Capture::Tiny at top level. The deb ships them on every architecture and
declared nothing for it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
perl-xcat declared libhtml-form-perl and nothing else. Its files load
XML::Simple, IO::Socket::SSL, DBI, JSON, LWP, XML::LibXML, Expect and SNMP
at top level, and every xCAT command goes through Client.pm, which loads
two of them and makes XML::Parser the XML::Simple parser. On a management
node xcat-server and the metapackage declared them, so the gap showed only
on a client-only install. dh_perl adds no module dependencies, unlike the
rpm generator.
Declare them where they are loaded. Socket6 and IO::Socket::INET6 are
declared too: their loads are guarded, but without them xcatd, the client,
SLP, nodestat, IPMI and getipaddr have no IPv6, and the Kea DHCPv6
reservation builder receives an IPv4 address for a node.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The plugin is loaded and install_darch is called for the architectures
xCAT installs Ubuntu on and two it does not. Against the previous plugin the
test fails on the missing function: the mapping was inline in mkinstall.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
mkinstall mapped x86_64 and x86 to their Debian names and accepted ppc64le
and ppc64el. Every other architecture, riscv64 included, was logged as
"Unknown arch" on each diskful install, although the install went on with
the name unchanged, which is right for riscv64.
Move the mapping into install_darch, which takes the Debian name from
xCAT::Utils::debian_arch and knows the architectures xCAT installs Ubuntu
on. riscv64 is one of them.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The grub-common assertion fails against the previous metapackage. The rest pins
what must NOT change: a riscv64 management node still recommends the x86 boot
payload and the Genesis images of the other architectures, because it serves
them to the nodes it provisions.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
copycd builds the riscv64 boot loader by running grub-mkimage, which
grub-common ships on every supported Ubuntu release. Nothing declared it, so a
management or service node installed without that package copies riscv64 media
and produces no loader, while DHCP keeps pointing every riscv64 node at the path
where the loader should be.
The declaration belongs to xcat-server, which carries the plugin that runs the
command, so both metapackages inherit it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The stubbed test proves the decisions copycd makes; this one proves the
artifact. It runs the real grub-mkimage over the grub2 package of copied media
and checks the image against the validation nodeset depends on, so a package
layout change or a grub-mkimage that stops accepting these inputs is caught
where it happens.
It needs media and the grub2 tools, so it skips without
XCAT_TEST_UBUNTU_RISCV64_MEDIA. Run on a management node against the 24.04 and
26.04 riscv64 trees.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The mirror assertions passed a Debian architecture straight in and so never
exercised the conversion genimage performs first. They now start from the xCAT
osarch value, which is what let the 32-bit x86 token reach the wrong archive.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
genimage converts osimage.osarch with debian_arch and uses the result twice: it
picks the apt mirror and it becomes debootstrap --arch. The map knew x86_64 but
not x86, so an image with osarch=x86 selected the ports archive, which carries
no i386, and then asked debootstrap for an architecture it does not know.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Four assertions fail against the previous debian/control: the dependency has no
architecture restriction, and dpkg's own parser still reports it for riscv64.
The last assertion pins that the restriction drops nothing else.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
xcat-genesis-scripts-amd64 is Architecture: all, so the plain Depends installed
the x86 legacy Genesis scripts, and the x86 Genesis base with them, on a riscv64
management node. riscv64 has no legacy Genesis: its image ships as
xcat-genesis-openembedded-riscv64, which the metapackage already recommends and
mknb consumes. xCAT.spec makes the same distinction on the rpm side.
The dependency is now restricted to the architectures that have a legacy
Genesis. amd64 and ppc64el keep it unchanged.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The guides said riscv64 covered EL10 only, and the riscv64 page listed
Ubuntu as unsupported. Both support matrices now carry the architecture for
Ubuntu, and the riscv64 page describes the Ubuntu paths: the loader copycd
builds from the media, the installer needing none of the accommodations
EL10 requires, the ports archive the packages come from, and a management
node running on riscv64. The 26.04 media need the RVA23 profile, which is
recorded as a limitation.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The note shown when boot/grub2/grub2.<arch> is missing told the
administrator it comes from grub2-xcat or the EL installation media. On
Ubuntu neither is true: copycd builds the loader from the grub2 package on
the media, because the image the media carry cannot boot over the network.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The noderes.netboot table gave riscv64 the condition >=el10, so the only
documented way to reach grub2 on the architecture was an EL node, although
lookupNetboot answers grub2 for riscv64 whatever the operating system. The
condition now names Ubuntu as well, the way ppc64 names both of its
families.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
go-xcat stopped on riscv64 before it reached the package manager, so the
installer xCAT documents could not set up the management node the riscv64
packages are built for. The architecture is now accepted alongside the
others.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Calls the selection the template performs. Covers the ports archive for
ppc64el and both spellings of ppc64, the main archive for every x86
spelling, an unknown architecture keeping the previous default, and
site.ubuntu_apt_mirror overriding both without an empty value blanking the
mirror.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
archive.ubuntu.com publishes amd64 and i386 only, so a ppc64el or riscv64
node was given an apt mirror carrying no package for it and the installer
could not fetch what the minimal live media lacks. The default is now the
ports archive for those architectures, chosen from the osimage's
architecture rather than the package directory, which is whatever path the
administrator configured. site.ubuntu_apt_mirror still overrides it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Calls the escaping the plugin performs. Covers the Ubuntu installer seed
keeping the arguments after it, every separator grub2 recognizes, a command
line without one staying byte for byte the same, escaped and quoted values
surviving unchanged, a separator beside a quoted span in the same word, and
a variable reference being left alone so BOOTIF still expands.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
grub2 reads its configuration as a script, so an unquoted command separator
ends the linux command and everything after it is lost. The Ubuntu
installer seed is written as ds=nocloud-net;s=<url>, so the node booted
without the seed URL and without the arguments that followed it, including
BOOTIF. The installer then found no autoinstall configuration and waited
for someone to answer its questions. A separator that is neither escaped
nor inside a quoted span is now escaped where it stands, which grub2
removes before it hands the line to the kernel. A value the caller escaped
or quoted keeps exactly the form the caller gave it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Reads the list imgutils returns. Covers the drivers a node needs to reach
its root filesystem, the overlay module, the architectures that already
worked keeping theirs, and an unknown architecture still getting an empty
list.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The Ubuntu driver table had no riscv64 entry, so genimage was handed an
empty list and built an image carrying no network module. A node whose NIC
is not built into the kernel then has no interface to fetch its root
filesystem with. The architecture now gets the same drivers the enterprise
Linux table lists for it, plus the overlay module every Ubuntu image needs.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Evaluates the selection genimage performs, so the test tracks the script.
Covers the ports archive for ppc64el and riscv64, the main archive for
amd64 and i386, and site.ubuntu_apt_mirror overriding both without an
empty value blanking the mirror.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
archive.ubuntu.com publishes amd64 and i386 only, so debootstrap could not
find a single package for a ppc64el or riscv64 netboot image and genimage
failed on every architecture except x86. The default mirror is now the
ports archive for those architectures. site.ubuntu_apt_mirror still
overrides it, for a local mirror that serves every architecture.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Extends the builder tests with the per-package architecture lists and the
architectures a release declares, and adds the generated mklocalrepo.sh
mapping a riscv64 host to its own repository. Also checks that xcat and
xcatsn declare riscv64 without losing amd64 or ppc64el.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The xcat and xcatsn packages declared only amd64 and ppc64el, and the
builder built and indexed only those two architectures, so a riscv64
management node had no package to install. Both packages are now built for
riscv64, every release the repository serves declares the architecture,
and the generated mklocalrepo.sh maps a riscv64 host to its own repository
instead of amd64. Each package now carries its own architecture list:
xCAT-genesis-scripts keeps the two it has control files for, because
riscv64 Genesis ships as an OpenEmbedded package instead.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Drives the build with dpkg-deb and grub-mkimage shadowed by stubs, so the
assertions read the arguments that decide whether the image can boot over
the network: the firmware format, the prefix nodeset writes into, the
module directory, and the network modules. Also covers the warning a node
without a loader gets, an interrupted build leaving nothing behind, a real
riscv64 loader being kept, text, a truncated image, another architecture's
loader, images with nothing to execute and an image built for another boot
path all being replaced, and a rejected loader being gone when the media
cannot replace it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
riscv64 nodes have no boot loader unless one reaches /tftpboot/boot/grub2,
and nothing on an Ubuntu management node puts one there. The grub2 image
the media carry cannot serve: it holds a built-in configuration that looks
for the live filesystem, so a node that loads it drops to a grub prompt
instead of reading the configuration nodeset writes. copycd now builds a
netboot image from the grub2 package the media ship, and warns when it
cannot, because the node has no other source for one. An image already in
place is kept only when it is a whole executable image for the
architecture the firmware loads and carries the prefix this boot path
needs; one that is not is removed, so a rebuild that cannot run leaves
nodeset reporting a missing loader rather than serving an unusable one.
The media of every other architecture are untouched.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Pin that both releases carry a riscv64 list, that it holds what the x86_64 list
holds, and that it keeps the kernel and the NFS client a diskless node needs.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Daniel Hilst
Vinícius Ferrão
Vinícius Ferrão