A chainloaded second stage announces itself in the user class, option 77, and
must be handed a script URL rather than the loader it just ran -- otherwise it
chainloads itself forever and the machine never finishes booting.
RFC 3004 length-prefixes each string in option 77. Plenty of firmware sends the
bare string instead, and the same loader sends either depending on how it was
built, so both encodings have to be recognised. The Kea policy has always
accepted both. The ISC side compared only `option user-class-identifier =
"xNBA"`, which is the bare form, so a conforming loader booting against an ISC
management node loops -- and does so silently, since from the server's side
every exchange looks like a normal first-stage boot.
Put the test in one place, isc_xnba_user_class_test, and use it from both the
per-network architecture chain and the three per-node statements. The per-node
ones reach dhcpd through omshell, so the helper takes the quoting its caller
needs. `substring(option user-class-identifier, 1, 4)` skips the length byte;
option 77 is declared as a plain string, and the same construct is already used
by the onie_vendor branch a few lines below.
debian_sysconfig_interface_keys fell back to querying the local dpkg when it
was handed no version. The caller always passes one, so the fallback bought
nothing -- and it made the "version could not be established" case answer
differently depending on whether the machine running the code happened to have
isc-dhcp-server installed. It passed on a developer's EL workstation and failed
on the Ubuntu CI runner, which is the failure it was always going to produce.
An undefined version now means what the caller means by it: could not be
established, so write every spelling. Covered by a test that stubs the lookup
to prove the answer no longer depends on the machine.
makedhcp wrote the interface list into /etc/default/isc-dhcp-server as
INTERFACES="..."
which is no longer always the variable the daemon is started with. The
systemd unit sources that file and expands exactly one variable onto dhcpd's
command line, and which one changed with the package:
14.04 4.2.4-7ubuntu12 sysvinit only $INTERFACES
16.04 4.3.3-5ubuntu12 unit $INTERFACES
18.04 4.3.5-3ubuntu7 unit $INTERFACES
20.04 4.4.1-2.1ubuntu5 unit $INTERFACES
22.04 4.4.1-2.3ubuntu2 unit $INTERFACESv4
24.04 4.4.3-P1-4ubuntu2 unit $INTERFACESv4
26.04 4.4.3-P1-4ubuntu2 unit $INTERFACESv4
with the matching v6 unit reading $INTERFACES before the change and
$INTERFACESv6 after it. Note the boundary is not an upstream ISC release:
20.04 and 22.04 both ship upstream 4.4.1 and are told apart only by the
Debian revision, so it has to be decided on the whole package version.
The sysvinit script does copy INTERFACES into INTERFACESv4 when the latter is
empty, but the unit is what starts the daemon on any of these releases and it
has no such bridge. So from 22.04 on, the unit ran
exec dhcpd -user dhcpd -group dhcpd -f -4 ... -cf $CONFIG_FILE $INTERFACESv4
against a variable xCAT never set. An unset variable expands to nothing, so
dhcpd was launched with no interface argument at all. It does not fail for
that: it binds every interface it can find and only warns about the ones with
no subnet declaration. site.dhcpinterfaces and servicenode.dhcpinterfaces
were therefore silently inert -- the provisioning NIC was served because
makedhcp had also emitted a subnet stanza for it, not because anything
honoured the setting, and any other interface on that subnet was served
alongside it.
The line match compounded it. m/^$dhcpd_key/ is not anchored on the
assignment, and INTERFACES is a prefix of both variables the package ships.
Since 18.04 the postinst seeds only INTERFACESv4="" and INTERFACESv6="" --
no INTERFACES line exists to match -- so the rewrite claimed those two lines
instead and overwrote both, discarding whatever debconf or the administrator
had put there and leaving a file with two INTERFACES assignments and nothing
either unit reads.
Pick the variables through a dispatcher keyed on the installed
isc-dhcp-server version, so both behaviours are served rather than one being
traded for the other, and anchor the match on the '=' so a key cannot claim a
line it is merely a prefix of. When the version cannot be established the
dispatcher writes every spelling, since an unset variable is the outcome that
leaves dhcpd bound to everything. Duplicate assignments left behind by the
old writer are collapsed to one, which repairs a file already damaged on an
upgraded management node.
The EL and SLES paths keep their single DHCPDARGS / DHCPD_INTERFACE /
DHCPD6_INTERFACE key and are unchanged.
On Debian and Ubuntu, makedhcp writes the list of interfaces it is serving
into /etc/default/isc-dhcp-server using the key set at dhcp.pm:2296:
$dhcpd_key = "INTERFACES";
That key stopped being the one the daemon is started with. The variable the
isc-dhcp-server systemd unit expands onto dhcpd's command line changed with
the package; verified by unpacking the archive's own debs:
trusty 4.2.4-7ubuntu12 sysvinit only INTERFACES
xenial 4.3.3-5ubuntu12 unit $INTERFACES
bionic 4.3.5-3ubuntu7 unit $INTERFACES
focal 4.4.1-2.1ubuntu5 unit $INTERFACES
jammy 4.4.1-2.3ubuntu2 unit $INTERFACESv4
noble 4.4.3-P1-4ubuntu2 unit $INTERFACESv4 (+ a v6 unit
reading $INTERFACESv6)
and every package from bionic onward ships a default file whose only
interface variables are INTERFACESv4 and INTERFACESv6, both seeded empty by
postinst. No INTERFACES line ships at all.
Two things go wrong on jammy and later.
The unit runs `exec dhcpd ... -cf $CONFIG_FILE $INTERFACESv4`. That variable
is never set, so it expands to nothing and dhcpd is launched with no
interface argument. dhcpd does not fail for this: it binds every interface
it can find and merely warns about the ones with no subnet declaration. So
site.dhcpinterfaces and servicenode.dhcpinterfaces are silently inert. The
provisioning NIC is served only because makedhcp also emitted a subnet
stanza for it, not because anything honoured the setting, and any other
interface on that same subnet -- a bridge port, a bond member, a second NIC
-- is served too.
The line match is m/^$dhcpd_key/, which is not anchored on the assignment.
"INTERFACES" is a prefix of both variables the package ships, so the rewrite
claims the INTERFACESv4 and INTERFACESv6 lines and overwrites both. The file
is left holding two INTERFACES lines and nothing the units read, discarding
whatever debconf or the administrator had set there.
Extract the rewrite into _sysconfig_interfaces_content() with no change in
behaviour, and add a unit test that asserts the effect rather than the
spelling: it writes the produced file to a temp path and sources it with sh
exactly as the unit does, then checks what would land on dhcpd's command
line. The test is red against the current key and stays red until the writer
sets the variables the daemon is actually started with.
mkinstall mapped x86_64 and x86 to their Debian names and accepted ppc64le
and ppc64el. Every other architecture, riscv64 included, was logged as
"Unknown arch" on each diskful install, although the install went on with
the name unchanged, which is right for riscv64.
Move the mapping into install_darch, which takes the Debian name from
xCAT::Utils::debian_arch and knows the architectures xCAT installs Ubuntu
on. riscv64 is one of them.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The note shown when boot/grub2/grub2.<arch> is missing told the
administrator it comes from grub2-xcat or the EL installation media. On
Ubuntu neither is true: copycd builds the loader from the grub2 package on
the media, because the image the media carry cannot boot over the network.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
archive.ubuntu.com publishes amd64 and i386 only, so a ppc64el or riscv64
node was given an apt mirror carrying no package for it and the installer
could not fetch what the minimal live media lacks. The default is now the
ports archive for those architectures, chosen from the osimage's
architecture rather than the package directory, which is whatever path the
administrator configured. site.ubuntu_apt_mirror still overrides it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
grub2 reads its configuration as a script, so an unquoted command separator
ends the linux command and everything after it is lost. The Ubuntu
installer seed is written as ds=nocloud-net;s=<url>, so the node booted
without the seed URL and without the arguments that followed it, including
BOOTIF. The installer then found no autoinstall configuration and waited
for someone to answer its questions. A separator that is neither escaped
nor inside a quoted span is now escaped where it stands, which grub2
removes before it hands the line to the kernel. A value the caller escaped
or quoted keeps exactly the form the caller gave it.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
riscv64 nodes have no boot loader unless one reaches /tftpboot/boot/grub2,
and nothing on an Ubuntu management node puts one there. The grub2 image
the media carry cannot serve: it holds a built-in configuration that looks
for the live filesystem, so a node that loads it drops to a grub prompt
instead of reading the configuration nodeset writes. copycd now builds a
netboot image from the grub2 package the media ship, and warns when it
cannot, because the node has no other source for one. An image already in
place is kept only when it is a whole executable image for the
architecture the firmware loads and carries the prefix this boot path
needs; one that is not is removed, so a rebuild that cannot run leaves
nodeset reporting a missing loader rather than serving an unusable one.
The media of every other architecture are untouched.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The riscv64 live-server image keeps its kernel at casper/vmlinux, where every
other live image keeps casper/vmlinuz, so the probe found no kernel and mkinstall
reported that the install image was missing.
Add the riscv64 candidate pair and let copycd name the architecture the media
reports. Verified against Ubuntu-Server 24.04.4 riscv64, which carries
casper/vmlinux, casper/initrd and casper/install-sources.yaml.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
getcredentials answered xcat_secure_pw only for root, so a postscript
had no way to get the password of another node account from the passwd
table.
xcat_secure_pw:<user> now returns the password field of the passwd row
key=system,username=<user> when <user> is root or a sudoer named in the
postscripts or postbootscripts of the requesting node, its osimage, or
xcatdefaults. A sudoer without a row or without a password gets the
locked field "!", so the node applies the reply as is. Any other user,
an invalid user name, or a failed hash answers with an error instead of
an empty reply. The root request reads the same row as before and keeps
the error reply for a missing row.
copycd translated the architecture the Ubuntu media reports with its own
if/elsif chain, and genimage translates the same names back for debootstrap with
another one. Neither can be reused, so a new architecture has to be added to
both.
Put both directions in xCAT::Utils and have copycd read from there. The names
and the fallback do not change: media that xCAT has no name for still leave the
architecture as the media reported it.
The probe spelled out every candidate path twice inside one nested condition,
once to test it and once to assign it, so adding an architecture meant adding
another branch of the same shape. Move the candidates into a table keyed by
architecture family and walk it in order.
Same paths, same precedence, same failure behaviour: a media tree that matches
nothing leaves the caller on the "install image not found" path as before.
The boot flip in compute.subiquity.tmpl addressed port 3002. xcatd's install
monitor listens on site.xcatiport, so a cluster that moves the port loses the
flip and every node PXE-loops back into the installer. The flip now reads
site.xcatiport and keeps 3002 as the default. TABLEBLANKOKAY, because the key is
optional and a plain TABLE lookup of an absent key fails the whole template.
The flip also counted any reply as an accepted request. It now requires the
monitor's "ready" greeting before it sends "next", and "done" afterwards, so a
different service on that port is not read as a flipped node.
subiquity_nfsroot_server in debian.pm called getipaddr without a family. A
dual-stack management node answers with its IPv6 address, and casper takes
everything after the first colon in nfsroot= as the path, so the live filesystem
never mounts. It now asks for IPv4, as dhcp.pm and mknb.pm do.
The DNS setup wrote the xcatmaster name as a nameserver when getent found no
address, which is the case the step exists to prevent. It now keeps the
resolv.conf DHCP gave the live installer.
ubuntu_subiquity_boot_flip.t, debian_subiquity_boot_params.t and
ubuntu_resolvconf_ip.t fail on the parent commit and pass here.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
`makedhcp -q <node>` on Ubuntu's ISC-limited releases answers "no DHCP reservation
found" when it cannot read dhcpd.conf. The operator reads that as a node without a
reservation. An InfiniBand node also gets an answer with no hardware address.
_query_isc_static_host in dhcp.pm read the file with an -r test and dropped a failed
open. It also matched only a "hardware ethernet" line, while _add_isc_static_host
writes "hardware infiniband" for an InfiniBand node and adds a twin declaration
between the same markers.
_read_isc_conf_lines now returns the read error, _query_isc_static_host returns it to
listnode, and listnode answers the caller with an error. The parser accepts any
hardware type and keeps the first declaration of the block. The path of dhcpd.conf and
the distribution name are package variables, so a test can drive the query and
listnode.
dhcp_isc_static_host_query.t covers the InfiniBand address, the twin declaration, the
unreadable file and the listnode answer. It fails without this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xCAT::NTP::Backend->available reported chrony as available on chronyd alone, while makentp
configured chrony only where systemctl was present too. On a host with chronyd and no systemctl
the selector returned chrony with no downgrade, makentp fell through to the ntpd path, and the
admin saw either a silent switch or "Please make sure ntpd is installed".
available now requires chronyd and systemctl for chrony, so the selector answers on the same
terms makentp acts on, and makentp branches on the name alone. choose therefore downgrades to
ntpd, or reports install, in the case it used to pass over. A commands argument injects the
command probe, in the same shape as the existing available argument.
ntp_backend_selection.t covers both commands. Six of its assertions fail without this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Fall back to an available DHCP backend on auto-selection. When the request is "auto"
and the backend chosen for this OS is not installed, use the other one if it is,
recording fallback_from so process_request can tell the operator which preferred
backend is missing. A backend the admin forced through site.dhcpbackend still fails
hard when absent, and "neither installed" still errors clearly. Fixes#7710.
Answer makedhcp -q from the static host block on Ubuntu's ISC-limited releases.
listnode now branches on _isc_static_host_fallback() before any omapi work and reads
the node's fixed-address and hardware ethernet straight out of dhcpd.conf, so the
query path never spawns the omshell its own write paths already avoid. A node with no
reservation is now reported rather than answered with silence.
Match the host-block markers exactly. _add_isc_static_host writes a fully determined
pair -- "#xCAT host declaration for <node> aka host <hostname> start" and the "}"
line carrying the matching end -- so both scans anchor on that whole shape through
shared _isc_host_start_re/_isc_host_end_re helpers. The previous /\Q$node\E\b.*/ also
matched at a hyphen, letting node "compute" act on "compute-01"'s block: the query
could return another node's address and the delete could remove another node's
reservation. _delete_isc_static_host also accepts an explicit line list now, so the
scan is unit testable without file-scoped state.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The module overrides two methods of XML::Simple, one for a recent version and
one for an older version, and each built its own parser and set its own
handlers. The two bodies were the same apart from spacing, so a change to one
refusal had to be repeated in the other, and a reader had to compare them to
see that they agreed.
Build the parser in one routine that both call. Behaviour does not change.
Six modules wrote passwords to their own log and diagnostic messages,
outside the daemon redaction pipeline. The z/VM plugin logged each
smcli command line through printSyslog, with the disk read, write and
multi passwords, the image password, the provision root password and
the page volume parm disk password, passed the real disk passwords to
checkSSH_Rc, which echoes the command to syslog and to the client on
failure, and logged raw directory entries whose USER and MDISK
statements carry the logon and disk passwords. The bmcconfig plugin
logged the BMC password in its attribute report, in syslog and in the
command response. The energy plugin logged the HCP password in a
verbose message, and the CIM utilities dumped the whole HTTP request,
with its basic authorization header, to the verbose callback. The PPC
configuration module logged the HMC, FSP and BPA passwords in its
verbose credential reports.
Mask the passwords in the logged text. The executed commands keep the
real values. The page volume log string is built by operand position,
so a decoy value in another operand cannot divert the mask. The
checkSSH_Rc calls receive the masked command string, as the routine
documentation asks. Add redact_directory_entry to the z/VM utilities.
The routine masks the USER, IDENTITY and IDENT logon password, the
MDISK passwords after the access mode in the range form and in the
DEVNO, V-DISK and T-DISK forms, the APPCPASS statement, and the
keyword password assignments in the short and the full spelling. The
match separators stay on one line, so a record without passwords never
masks the record below it, and one or more comment stars do not hide a
credential record from the rules. The COMMAND statement masks whole,
because it can start any CP command with an inline password. Every directory query sink logs
through it,
and the clone loops redact the query output at the source, because the
failure checker and the retained disk list reuse the text. The
directory helpers keep their raw return value for the callers and hand
a redacted copy to the failure checker. Every error branch that echoes
a fetched record after the output check does so through the redactor,
because a password can spell an error word and trip the check: the
directory fetch, the mini disk keyword fetch, and the four disk list
callers. The CIM dump masks
the authorization header. The bmcconfig report now names the password
state, set or missing, which the report needs for diagnosis.
The previous commit deleted the $effective_provmethod override on the grounds
that %image_hash never carries a provmethod. That was wrong, and the review
caught it: makescript builds %image_hash, calls getImage() on it, and then
hands the SAME hashref to getScripts(), which fills provmethod for every
osimage from the osimage table. getDisklessNet() already reads that key the
same way. The override was live, not dead.
Restore it and say what is actually true in the comment. nodetype.provmethod is
frequently an osimage name rather than 'install', and resolving it is the point
of the lookup.
Also close the gap that made the wrong deletion so easy to ship: reverting
mkinstall's subiquity branch to its pre-fix body left the whole unit suite
green. debian_mkinstall_subiquity_branch.t lifts that branch out and drives it
inside a real loop, so the `next` it performs is the one under test, with
report_node_error and the getipaddr seam stood in for. It calls
subiquity_boot_params with no injected resolver, exactly as production does.
The branch is selected out of debian.pm by what it contains rather than by
where it sits -- there are four `if (using_subiquity(...))` in that file, and an
earlier draft of this test silently matched the wrong one and ran past its
block.
Now observable, each verified by mutation: swapping $pkgdir and $httpport at
the call site reddens the nfsroot assertion; reverting the branch wholesale
fails the extraction guard rather than passing.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The daemon reads the XML of every request through this parser. A request could
declare an entity in its own document type declaration, and the parser expanded
it. An entity that refers to other entities grows on each level, so a short
request expands into a large document and consumes the memory and the time of
the daemon. A client holds a certificate before it can send a request, so this
needs an account, but the daemon should not accept the work.
Refuse the declaration itself. The option that stops the parser from expanding
an entity does not cover an entity that a request names inside an attribute, so
it leaves the same growth available through a different part of the document.
Measured on XML::Parser 2.46, a request of 204 bytes that names its entity in
an attribute still grew to 1014 bytes with that option set, which is what the
parser does without it.
No request that xCAT sends carries a document type declaration. The client
builds every request with XML::Simple, which does not write one.
The handler that refuses an external entity stays, so a parser that reaches it
by another route still refuses to read the named file.
Both parser constructors passed a list of options to XML::Parser as an array
reference:
XML::Parser->new(Style => 'Tree', [ load_ext_dtd => 0, ... ]);
XML::Parser->new takes a flat list of pairs. The reference is one value in that
list, so the constructor reads the pairs as Style => 'Tree' and then the
reference as the name of an option with no value. Every option inside the
reference is dropped. The names are also the names that XML::LibXML uses, not
the names that XML::Parser uses, so the parser would ignore them even if it
received them.
The options therefore never did anything, and they give the reader the
impression that the parser refuses an external entity because of them. The
handler on the next line is what refuses an external entity.
Remove them. Behaviour does not change.
The syncfiles deferral resolved the node's provmethod through
$image_hash{$osimgname}{provmethod} when the node names an osimage. makescript
fills %image_hash from getImage(), which stores pkglist, pkgdir, otherpkglist,
otherpkgdir and environvar -- and no provmethod. getScripts() has a separate
hash that does store one, which is where the pattern was copied from. So the
lookup was always undef, the override never fired, and the code claimed a
behaviour it did not have.
Pass $provmethod directly and say in the comment why there is nothing to
resolve it with. No behaviour changes -- the branch was inert -- so there is no
red to show first; what the deletion needs is coverage that the path it was
supposed to serve still works.
That is what the two new assertions do: an osimage-named provmethod with
nodesetstate 'install' still defers, and the same name with no nodesetstate is
not mistaken for a diskful install. nodesetstate is what carries the install
signal here, which is why the override was never load-bearing. Making the
deferral ignore nodesetstate and require provmethod eq 'install' reddens both.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Two things the review found, both in code added by this branch.
Reverting mkinstall's call site -- putting xCAT::NetworkUtils->getipaddr back
in place of subiquity_nfsroot_server, the exact regression the fix removes --
left the entire unit suite green. The helpers were covered; nothing linked them
to production. Compose the two steps in subiquity_boot_params(), which takes its
inputs and returns either a command line or the reason there isn't one, so the
composition can be driven; mkinstall keeps report_node_error and the loop's
`next`. That same revert now reddens 6 of 9 assertions.
The test stubs xCAT::NetworkUtils::getipaddr deliberately. Without it a call
site that bypassed the injected resolver died on a missing module -- a red, but
for the wrong reason. With it, bypassing the resolver returns the wrong answer,
which is what the assertions are there to catch.
The resolv.conf sandbox guard matched `/etc/` with a trailing slash, so the one
respelling its own comment names -- `etcdir=/etc; rm -f "$etcdir/resolv.conf"`
-- walked straight past it and the fragment would rm the runner's real
resolv.conf, as root in CI. `/etc\b` catches it: applying that respelling now
BAIL_OUTs instead of running.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The NTP backend selector was well covered and nothing connected it to makentp:
copying the base makentp.pm over the head one left the whole unit suite
byte-identical, so the branches that consume choose()'s answer -- abort on a
selector error, warn on a downgrade, abort when neither daemon is installed --
and the --backend argument handed to setupntp were covered by nothing.
They were unreachable from a test because they sat inside process_request,
which needs a management node. Move the decisions into ntp_backend_action() and
setupntp_command(), which take their inputs and return an answer; the caller
keeps send_msg and runcmd. No behaviour changes -- the same messages are sent
on the same conditions, and the same command is built.
Verified by mutation rather than by reading: dropping the install abort reds 3
of 15, the downgrade note 1, the --backend argument 2, the server-list split 1,
and the selector-error abort 2.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Daniel Hilst
Vinícius Ferrão