The step that fills lib/firmware in the Ubuntu netboot initrd asked modinfo
about $rootimg_dir/$module and looked a firmware name up under lib/firmware.
The copy step beside it takes a module from $customdir or $pathtofiles first,
and the kernel looks a firmware name up under updates/<kernel>, updates/,
<kernel>/ and lib/firmware. So a custom driver reached the initrd with no
firmware, even when the root image carried it, and a firmware override was left
out of the initrd altogether.
initrd_firmware_files now takes the module directories the copy step searches
and the kernel release. It resolves each module in that order before asking
modinfo, and keeps every firmware file that exists in the four directories the
kernel searches, so the override still wins on the node.
ubuntu_genimage_initrd_firmware.t covers both: its two new cases are red on the
commit before this one.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
makedns reports "error was FORMERR" for an update that named rejected with
NOTAUTH. The FORMERR is the answer to the retry, not to the first attempt.
send_ddns_update in ddns.pm signs the packet the caller built, and signs that
same packet again on each attempt. Net::DNS::Packet::sign_tsig appends the TSIG
to the additional section, so the second attempt sends two TSIG records and
named answers FORMERR. FORMERR is neither NOTAUTH nor SERVFAIL, so the routine
stops and reports it. The NOTAUTH and SERVFAIL retry can never be accepted, on
any algorithm.
Each attempt now signs a request of its own. A packet cannot be unsigned again,
so ddns_update_request copies the prerequisite and update records into a new
Net::DNS::Update instead, and the caller keeps the unsigned original.
ddns_update_retry.t fails before this change: the second attempt carries two
TSIG records, and an update that the retry answers with NOERROR still reports
failure.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
makedns exits 1 on a management node that has Net::DNS below 1.36 and an
hmac-sha256 key, and reports "Failure encountered updating <zone> with entry
'', error was FORMERR".
update_namedconf in ddns.pm rewrites the named.conf key stanza to hmac-md5
whenever Net::DNS is below 1.36, and ddns_tsig_algorithm returns hmac-md5 for
the same reason. ddns_sign_update signs with site.dhcpomapialgorithm, which
xcatconfig sets to hmac-sha256 on EL9 and later. named matches a TSIG key by
name and by algorithm, so it answers NOTAUTH. The retry signs the same packet
a second time, and named answers FORMERR to the two signatures.
The version test protected the two-argument sign_tsig($name, $secret), which
produces an HMAC-MD5 signature only. ddns_sign_update signs every other
algorithm through a KEY RR, so the Net::DNS version no longer selects the
algorithm. This change deletes the rewrite and the version test, and signs with
the algorithm the key stanza declares. OmapiPolicy->algorithm_rr_type maps that
algorithm to its KEY RR number.
ddns_named_key_algorithm.t fails before this change: it reads the stanza as
hmac-md5 where the key was hmac-sha256.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The ubuntu-26.04-ppc64el diskless node never starts its kernel. SLOF reports
W3411 and E3406 and falls through to disk. The node fetches the kernel and the
initrd in under a minute and fails six minutes later, with no network activity in
between: grub2 has the payload and cannot start it.
genimage copies the whole lib/firmware tree of the root image into the initrd.
That tree is 666 MB on Ubuntu 26.04, which takes the initrd to 719 MB. A 687 MB
initrd boots the same kernel on the same node; a 719 MB one does not.
The firmware copy now takes only the firmware that the drivers in the initrd ask
for, which modinfo reports for each module. The root image keeps its whole tree,
so the node that boots is unchanged.
ubuntu_genimage_initrd_firmware.t drives the firmware step over a root image whose
firmware tree holds files no driver asks for. It fails on the previous commit.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
genimage returns 2 with "Failed to find usr/bin/dig" for
ubuntu22.04 and ubuntu24.04 on ppc64el, so packimage writes no initrd.gz
and the diskless compute node never boots.
xCAT-server/share/xcat/netboot/ubuntu had a ppc64el package list for
20.04 and 26.04 only. imgutils::get_profile_def_filename then falls back
to compute.pkglist, which installs no dig, no cpio and no chrony.
Add the 22.04 and 24.04 ppc64el lists, and the ppc64le spelling each
release already carries. Both take the content of the 26.04 ppc64el list:
the ppc64el images build their initrd with mkinitrd, so they install
bind9-dnsutils and leave out the dracut packages the x86_64 lists take.
ubuntu_ppc64el_pkglists.t fails without these files.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
reg_linux_diskfull_installation_flat fails on ubuntu-22-ppc64le-devel and
ubuntu-24-ppc64le-devel in build #121 of xcat-core-devel-ubuntu-cd. The
Subiquity installer starts, errors in its early-commands, tars
/var/log/installer to port 8080 and reboots, nine times in 75 minutes. No
system is ever installed, so the address answers from the live installer and
the case ends on
"root@xcat25-cn: Permission denied (publickey,password)".
mkinstall in xCAT-server/lib/xcat/plugins/debian.pm selected
pre.ubuntu.subiquity and then replaced it with pre.ubuntu.ppc64 for every
ppc64 node, whichever installer was in use. pre.ubuntu.ppc64 writes a partman
recipe, and the early-commands append it to /autoinstall.yaml, which Subiquity
cannot parse.
pre.ubuntu.subiquity offered a UEFI branch and a BIOS branch, so a ppc64el node
took the BIOS branch and was given a bios_grub partition. POWER firmware loads
the boot loader from a PReP partition.
install_prescript now returns the script from the installer and the
architecture together, and the ppc64 script is reached only on the
debian-installer path. pre.ubuntu.subiquity gains a PReP branch, taken when
uname reports a POWER machine, which flags an 8M first partition prep and makes
that partition the grub device.
debian_install_prescript.t and the PReP case of ubuntu_subiquity_storage.t fail
without these changes.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
On Ubuntu 26.04 the installed compute node keeps systemd-resolved, which takes
the DNS server from DHCP and no search domain, so /etc/resolv.conf reads
"search .". mypostscript then runs "updateflag.awk $MASTER 3002" with the short
management node name, the flag update fails eleven times, and the node stays at
postbooting until retry_install.sh gives up. The management node does offer
domain-search; the node discards it.
compute.subiquity.tmpl writes /target/etc/netplan/00-xcat-install.yaml with
dhcp4: true alone, so systemd-networkd applies its UseDomains default of no.
Add dhcp4-overrides: use-domains: true to both branches, the one that renames
the interface and the one that matches by MAC alone.
ubuntu_subiquity_installnic.t runs the template's own late-command and asserts
the netplan it writes carries the setting. It fails without this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
master moved 149 commits ahead of the branch point and four files needed a
decision.
xCAT/debian/control and xCATsn/debian/control: master moved nmap and
ipmitool-xcat into Depends, raised the ipmitool version and added the s390x
OpenEmbedded Genesis recommendation. The branch made the genesis-scripts
dependency per architecture. Both are kept, so the ppc64el metapackage depends
on xcat-genesis-scripts-ppc64el and no longer on the amd64 package.
build-utils/lib/XCAT/BuildUtils.pm and xCAT-test/unit/build_utils.t: master
replaced @DEB_ARCHES plus the branch's %NO_RISCV64 exception list with
%ARCH_PACKAGES, which carries the architecture list per package.
deb_package_arches returns the same answer for every package, so master's form
is kept and %NO_RISCV64 is dropped.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
nodeset stops the diskful install on every ppc64le cell of
xcat-core-devel-ubuntu-cd (build #119, ubuntu-24-ppc64le-devel):
xcat25-cn: The network boot initrd.gz is not found in
/install/ubuntu24.04.4/ppc64el/install/netboot.
The Ubuntu ppc64el live-server ISO carries no netboot tree. The installer
kernel and initrd sit under casper. Two places in
xCAT-server/lib/xcat/plugins/debian.pm reject that media: %INSTALL_BOOT_FILES
describes no casper layout for POWER, and mkinstall applies a second,
POWER-only precondition on install/netboot/initrd.gz before it calls the
resolver.
The POWER table now carries casper/hwe-vmlinux + casper/hwe-initrd and
casper/vmlinux + casper/initrd, after the netboot entries so a netboot tree
still wins. install_media_is_bootable replaces the POWER-only precondition, so
one routine decides what media can boot.
debian_install_boot_files.t resolves the POWER casper layouts and drives
install_media_is_bootable. It fails without this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Ubuntu 20.04, 22.04 and 24.04 resolved the shared compute.pkglist on
every architecture but x86_64 20.04, and that list names ntp for the
releases before Subiquity. The Subiquity template installs chrony, and on
these releases ntp pulls ntpsec, which conflicts with it, so one apt
transaction with both cannot be satisfied and ospkgs replaced chrony
after the first boot.
Each of the three releases now has its own default list with chrony, the
list 26.04 already had. The shared list keeps ntp for the releases where
chrony was not the default.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
compute.subiquity.tmpl named a fixed package set, so the osimage pkglist
took effect only when ospkgs ran after the first boot. The packages list
now ends with the autoinstall token, so the pkglist packages install from
the configured apt mirror during the autoinstall. The fixed set stays, so
a node installs the same packages as before plus its pkglist.
The apt sources the installer gets for the pkgdir mirrors and the
otherpkgs repository are removed from the target at the end of the
install. ospkgs and otherpkgs write their own after the first boot, and a
second source for one repository with other options makes apt refuse the
whole list.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
A Subiquity autoinstall installs what its user-data packages list names,
and the templates could only name a fixed set, so the osimage pkglist
reached an Ubuntu node through ospkgs after the first boot. The preseed
token has no autoinstall form: the package list is YAML, one item per
line.
A list line that carries #INCLUDE_DEFAULT_PKGLIST_AUTOINSTALL# is now
replaced by one item per pkglist package at the same indentation, with
includes followed and without repeating the items the template lists
above it. The line is replaced in the include pass of subvars, so a site
template that includes the stock one is served too. A plain name and a task are
installed this way, and a comment after them ends the record. A version
pin or a target release stays with ospkgs, because the installer runs
apt-get without --allow-downgrades and a pin can require one, and so does
a name with an architecture qualifier, because a foreign architecture is
enabled by a postscript that runs later. A record that begins with
a removal or a group is left out whole, as ospkgs removes or installs it
whole, and so are a removal written with a trailing hyphen, markers and
preseed directives. A list that carries a #ENV: setting or an unreadable
include is left to ospkgs whole. So is the list of an osimage with
environvar, which mkinstall now hands over, and such an image's pkgdir
mirrors stay out of the installer's sources as well: those variables
reach apt-get only through ospkgs, and a mirror may need them. An osimage
without a pkglist loses only the token line.
The installer's apt configuration turns recommended packages off, as
ospkgs installs the list without them; curtin writes that setting into
the target, where the template removes it with the installer's sources.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
get_pkglist_tex joins the records of a pkglist with commas for the preseed
and OSPKGS consumers, so a caller that needs the records themselves cannot
recover a record that contains a comma, such as a tasksel directive. The
new reader returns the records whole, comments dropped and includes
followed in place. It reads each line as get_pkglist_tex does and resolves
every include, nested ones too, against the directory of the listed
pkglist, as get_pkglist_tex does. The comma text is unchanged.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
An osimage pkgdir can name mirrors after the install media, and ospkgs
installs from all of them, but the autoinstall apt configuration offered
the installer only the archive mirror and the otherpkgs repositories. A
package that only a pkgdir mirror carries could not be installed before
the first boot.
The mirrors now join the installer's apt sources, next to the otherpkgs
ones, in every form the configuration takes. An entry written as URL,
suite and components is an apt source line, as ospkgs writes it, and a
suite that is an exact path needs no component. A local directory that
is a flat repository is served by the management node and trusted, as an
otherpkgdir is. An entry that names an Ubuntu archive mirror the
installer already has a source for, the configured one or a default one,
carries that source's signing key, the archive keyring on the Deb822
releases and none before them, because apt rejects a second source for
the same suite whose signing key differs. For the same reason a
repository is offered once, whether pkgdir names it twice, as a directory
and as its URL, or the otherpkgdir names it too. Anything
else is no apt source for ospkgs either and is left out.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The online autoinstall apt configuration offered every otherpkgdir entry
as a flat one-line source with trusted=yes. From 24.04 on curtin
converts a one-line source to Deb822 before it writes the file and keeps
only the type, URI, suite and components, so the unsigned repository
reached apt without the option and apt rejected it during the install.
An entry written as URL, suite and components, the form the otherpkgs
documentation gives for a mirror, was written whole as the URL, which
apt cannot parse either.
The sources are now Deb822 stanzas on those releases, which curtin
writes as they are, with Trusted: yes, and a mirror entry keeps its
suite and components as fields. The releases before 24.04 keep the
one-line form, with the same fields.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Both copies wrote the capture to /tmp/dhcpdumpfile.log, so a file left by
another user or an earlier root run blocked the probe and two runs
overwrote each other. tcpdump ran behind a shell, so the script killed it
by searching the process table for any tcpdump on the interface, and a
tcpdump that failed to start left an empty file that read as zero DHCP
servers. An interrupt left the capture running.
The capture file is now a private temporary file removed on every exit.
The child execs tcpdump itself, so the script stops and reaps exactly that
pid, and a tcpdump that ended before the window did fails the run. A
failed send and an INT or TERM stop the capture and exit 1 as well.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
xcat-server ships xcatws.cgi and restapi.pl with the endpoint enabled in
apache, and both load CGI at top level. Nothing declared the module, so on
a management node where nothing else pulled it every /xcatws request
answered 500. The rpm package requires perl(CGI).
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
mkinstall mapped x86_64 and x86 to their Debian names and accepted ppc64le
and ppc64el. Every other architecture, riscv64 included, was logged as
"Unknown arch" on each diskful install, although the install went on with
the name unchanged, which is right for riscv64.
Move the mapping into install_darch, which takes the Debian name from
xCAT::Utils::debian_arch and knows the architectures xCAT installs Ubuntu
on. riscv64 is one of them.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
copycd builds the riscv64 boot loader by running grub-mkimage, which
grub-common ships on every supported Ubuntu release. Nothing declared it, so a
management or service node installed without that package copies riscv64 media
and produces no loader, while DHCP keeps pointing every riscv64 node at the path
where the loader should be.
The declaration belongs to xcat-server, which carries the plugin that runs the
command, so both metapackages inherit it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The note shown when boot/grub2/grub2.<arch> is missing told the
administrator it comes from grub2-xcat or the EL installation media. On
Ubuntu neither is true: copycd builds the loader from the grub2 package on
the media, because the image the media carry cannot boot over the network.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
go-xcat stopped on riscv64 before it reached the package manager, so the
installer xCAT documents could not set up the management node the riscv64
packages are built for. The architecture is now accepted alongside the
others.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
archive.ubuntu.com publishes amd64 and i386 only, so a ppc64el or riscv64
node was given an apt mirror carrying no package for it and the installer
could not fetch what the minimal live media lacks. The default is now the
ports archive for those architectures, chosen from the osimage's
architecture rather than the package directory, which is whatever path the
administrator configured. site.ubuntu_apt_mirror still overrides it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
grub2 reads its configuration as a script, so an unquoted command separator
ends the linux command and everything after it is lost. The Ubuntu
installer seed is written as ds=nocloud-net;s=<url>, so the node booted
without the seed URL and without the arguments that followed it, including
BOOTIF. The installer then found no autoinstall configuration and waited
for someone to answer its questions. A separator that is neither escaped
nor inside a quoted span is now escaped where it stands, which grub2
removes before it hands the line to the kernel. A value the caller escaped
or quoted keeps exactly the form the caller gave it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The Ubuntu driver table had no riscv64 entry, so genimage was handed an
empty list and built an image carrying no network module. A node whose NIC
is not built into the kernel then has no interface to fetch its root
filesystem with. The architecture now gets the same drivers the enterprise
Linux table lists for it, plus the overlay module every Ubuntu image needs.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
archive.ubuntu.com publishes amd64 and i386 only, so debootstrap could not
find a single package for a ppc64el or riscv64 netboot image and genimage
failed on every architecture except x86. The default mirror is now the
ports archive for those architectures. site.ubuntu_apt_mirror still
overrides it, for a local mirror that serves every architecture.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
riscv64 nodes have no boot loader unless one reaches /tftpboot/boot/grub2,
and nothing on an Ubuntu management node puts one there. The grub2 image
the media carry cannot serve: it holds a built-in configuration that looks
for the live filesystem, so a node that loads it drops to a grub prompt
instead of reading the configuration nodeset writes. copycd now builds a
netboot image from the grub2 package the media ship, and warns when it
cannot, because the node has no other source for one. An image already in
place is kept only when it is a whole executable image for the
architecture the firmware loads and carries the prefix this boot path
needs; one that is not is removed, so a rebuild that cannot run leaves
nodeset reporting a missing loader rather than serving an unusable one.
The media of every other architecture are untouched.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
24.04 and 26.04 had no riscv64 package list, so a diskless image or an install
for the architecture fell back to the generic list and reached debootstrap
without a kernel or the tools the boot scripts call.
The lists hold the same packages as their x86_64 counterparts. Every one of
them is published for riscv64 in noble and resolute, main or universe.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The image carries the name service libraries of its architecture, and riscv64
matched neither the x86_64 nor the ppc64el branch. It fell through to the
generic path, which looks for lib/libnss_dns.so.2, so a riscv64 image shipped
without a resolver and the node could not resolve any name.
Ubuntu keeps them in lib/riscv64-linux-gnu, confirmed in the 24.04.4 riscv64
server filesystem.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The riscv64 live-server image keeps its kernel at casper/vmlinux, where every
other live image keeps casper/vmlinuz, so the probe found no kernel and mkinstall
reported that the install image was missing.
Add the riscv64 candidate pair and let copycd name the architecture the media
reports. Verified against Ubuntu-Server 24.04.4 riscv64, which carries
casper/vmlinux, casper/initrd and casper/install-sources.yaml.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The go-xcat shell behavior tests were written as Perl harnesses, which made the shell assertions harder to read and kept shell-specific setup outside a native shell test framework.
Add BATS to the GitHub Actions dependency set, run BATS tests from the same preserved source tree as the Perl unit suite, and move the go-xcat repository checks into xCAT-test/autotest/bats.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Both copies refused to run unless /usr/sbin/tcpdump existed. Debian and Ubuntu
install it as /usr/bin/tcpdump, so the rogue DHCP detector never ran there and
the probe reported its tcpdump check as failed.
Resolve tcpdump through PATH and the standard system directories with
CommandUtils::find_executable, run the resolved path, and match that path when
the capture process is killed at the end.
The shared Ubuntu lists serve every release and architecture without a list
of its own, and ospkgs hands the whole list to one apt-get install, so one
unknown name loses every package on it. ntp and ntpdate are gone from 26.04.
libodbc1 is the unixODBC runtime name up to 22.04. libvirt-bin is gone from
20.04 on and qemu-kvm from 22.04 on.
The shared lists keep ntp and ntpdate, so releases up to 24.04 keep the
daemon they had. 26.04 gets release lists that carry chrony. unixodbc
replaces libodbc1 on every release, and the odbcsetup postscript keeps its
runtime. The service lists add libdbd-pg-perl beside libdbd-mysql-perl, as a
service node may run the xCAT database on PostgreSQL. libvirt-daemon-system
with libvirt-clients replaces libvirt-bin from 18.04 on; 12.04, 14.04 and
16.04 keep kvm lists with the old names.
qemu-kvm was a transitional name for the emulator of the host architecture,
and no current release has one name for that. Per-architecture kvm lists
name the native one: qemu-system-x86 on x86_64, qemu-system-ppc on ppc64el,
qemu-system-misc on 24.04 riscv64, with ppc64le linked to ppc64el as the
other lists do. The shared kvm lists fall back to
qemu-system, which carries every emulator, so an architecture without a list
of its own still gets one. On 26.04 riscv64 libvirt-daemon-system depends on
qemu-kvm or qemu-system and nothing provides qemu-kvm, so apt installs that
fallback there whatever the list names. Every kvm list names qemu-utils:
ospkgs installs without recommends, and libvirt needs qemu-img for the qcow2
volumes kvm.pm creates.
The check ran on EL9 only, and only when the version carried a minor number,
so CentOS Stream was never checked. On EL10 a management node without EPEL
or CRB failed inside dnf install with a dependency error instead of the
message that names the missing repository. The CRB message proposed a CentOS
Stream repository file with signature checks disabled, on every
distribution. The probe used dnf list, which an installed copy of the probe
package satisfies with the repository disabled, and which reports a failed
query as a missing repository.
Check EL9 and EL10, with or without a minor version. Probe the enabled
repositories with repoquery for the host architecture and noarch, so a
source repository does not stand in for the binary one, and stop with the
package manager's own error when the query fails. Name the EPEL release
package of the running major version. For CRB, name crb enable from a
current epel-release, which handles Red Hat Enterprise Linux under both
subscription management and RHUI, Rocky Linux, AlmaLinux and CentOS Stream,
and the dnf config-manager command for Oracle Linux.
On ppc64el "go-xcat install" and "go-xcat uninstall" stop with
E: Unable to locate package xcat-genesis-scripts-ppc64
apt refuses the whole transaction, so no package of the list is installed
or removed.
The dpkg branch of GO_XCAT_INSTALL_LIST in
xCAT-server/share/xcat/tools/go-xcat names xcat-genesis-scripts-ppc64 and
xcat-genesis-base-ppc64. Those are rpm architecture names.
GO_XCAT_UNINSTALL_LIST is derived from the same array, so both actions
carry the wrong names. The Debian architecture is ppc64el.
This change renames the two entries of the dpkg branch. The rpm branch
keeps ppc64, which is the Genesis target architecture of the spec.
go_xcat_genesis_package_names.t fails the four deb assertions without
this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Daniel Hilst
Vinícius Ferrão
Vinícius Ferrão