mirror of
https://github.com/xcat2/xcat-core.git
synced 2026-09-04 12:07:56 +00:00
Daniel Hilst
b8510e1be3
build-ubunturepo was 710 lines of shell doing the Debian half of what
buildrpms.pl does for rpms, with no code in common and a different CLI. It also
carried paths that are dead: GSA uploads, the PROMOTE/PREGA release flows, and a
-d mode that built an xcat-dep repository from a different project's packages.
builddebs.pl replaces it and mirrors buildrpms.pl -- Getopt::Long options, one
package list, build then index then sign -- so the two builders read the same way
and share BuildUtils.pm.
The design rests on one fact: xcat-core debs are Perl. They are byte-identical
for every Ubuntu release, so they are built ONCE and the same files are published
into every codename. Only xCAT, xCATsn and xCAT-genesis-scripts carry an
architecture, and there the difference is packaging metadata, not compiled
output. That is why this needs no sbuild and no per-codename chroot -- unlike
xcat-dep, whose packages are compiled and genuinely differ per release.
BuildUtils.pm holds what both builders need and what was worth making testable:
the Version-Release derivation from the commit time, the xCAT-probe helper
staging, the deb arch and dist tables, the debian/control version pinning, the
changelog rewrite, the reprepro conf generation, and the build lock. Every
function is pure or takes its side effect as an argument, so build_utils.t (45
assertions) drives each one rather than grepping a builder for evidence that it
is called. Verified by mutation: shrinking the arch table reddens 1, dropping
the /g from the control pin reddens 2.
The env-var CLI maps to options: BUILDALL=1 -> --force, GPGSIGN=1 -> --gpg-sign,
GPG_HOME -> --gpg-home, DEST -> --dest, DISTS -> --dist (repeatable). UP=0 has no
equivalent because uploading is gone -- the CD pipeline's deploy step publishes.
Callers updated: github_action_xcat_test.pl and travis.pl. The comment in
github_action_xcat_test.pl explaining why CI copies the tree before building is
corrected -- build-ubunturepo rm -rf'd $curdir/../../xcat-core, which under
GitHub's work/<repo>/<repo> layout is the checkout's own parent; builddebs.pl
writes under dist/debs inside the checkout and restores every file it edits, so
the copy is now only isolating the tests from build residue.
Two tests moved with it. build_ubunturepo_lock.t extracted the lock out of the
shell with a regex and ran that; the lock is now a function, so builddebs_lock.t
calls it -- and asserts what actually matters, that two builds of one checkout
fail fast while two builds of different checkouts run concurrently.
ubuntu_2604_pkglist.t asserted that resolute appeared in a shell fragment of
build-ubunturepo's source; it now asks BuildUtils for the release list and checks
a resolute stanza reaches conf/distributions. That assertion would have passed on
any file containing the fragment and broken on a reflow that changed nothing.
Verified: prove -r xCAT-test/unit fails on 6 files here against 7 on
upstream/master, the difference being apache_config_sources.t, fixed by the
preceding commit. The remaining 6 are missing DB modules on the machine that ran
it and are identical on both.
NOT done here, and required before this can merge: the Ubuntu core CD pipelines
still invoke ./build-ubunturepo (ci/ubuntu/Jenkinsfile.core-ubuntu-{devel,stable}
in VersatusHPC/xcat-core-ci-cd, and the inline script in each live Jenkins job).
Those must be switched to builddebs.pl in the same change window.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
115 lines
4.6 KiB
ReStructuredText
115 lines
4.6 KiB
ReStructuredText
Building Source Code
|
|
====================
|
|
|
|
xcat-core
|
|
---------
|
|
|
|
Clone the xCAT project from `GitHub <https://github.com/xcat2/xcat-core>`_ and
|
|
build the rpms with ``buildrpms.pl``::
|
|
|
|
cd xcat-core
|
|
./buildrpms.pl --target alma+epel-9-x86_64
|
|
|
|
Each package is built in its own ``mock`` chroot, so the build does not depend on
|
|
what happens to be installed on the build host. Pass ``--target`` once per target
|
|
to build several; the default is every supported EL target. ``./buildrpms.pl
|
|
--help`` lists the rest.
|
|
|
|
To build the source rpms and no binary rpms, pass ``--source-only``::
|
|
|
|
cd xcat-core
|
|
./buildrpms.pl --target alma+epel-9-x86_64 --source-only
|
|
|
|
A source rpm is the input that a build service such as mock, koji, COPR or OBS
|
|
takes, and it lets one machine make the source rpms while another makes the
|
|
binary rpms for each architecture. ``rpmbuild`` does not need the packages named
|
|
in ``BuildRequires`` to make a source rpm, so this also builds on a machine that
|
|
cannot complete a full build.
|
|
|
|
The source rpms land in ``dist/<target>/rpms/SRPMS/``. The binary repository
|
|
metadata is left as the last full build wrote it, and no ``.repo`` file is
|
|
emitted, because a source-only run has no binary packages to advertise.
|
|
|
|
.. note::
|
|
|
|
``buildcore.sh``, ``makerpm``, ``buildlocal.sh`` and ``build-ubunturepo``
|
|
were removed in 2.19. ``buildrpms.pl`` replaces the first three, and its
|
|
``--source-only`` replaces the old ``SRCONLY=1``; ``builddebs.pl`` replaces
|
|
``build-ubunturepo``.
|
|
|
|
Debian and Ubuntu packages
|
|
--------------------------
|
|
|
|
Build the ``.deb`` packages and an apt repository with ``builddebs.pl``::
|
|
|
|
cd xcat-core
|
|
./builddebs.pl
|
|
|
|
The packages land in ``dist/debs/debs/`` and the repository in
|
|
``dist/debs/xcat-core/``. Pass ``--dest`` to write them elsewhere, ``--dist`` to
|
|
limit which Ubuntu releases the repository serves, and ``--gpg-sign`` (with
|
|
``--gpg-home``) to sign it. ``./builddebs.pl --help`` lists the rest.
|
|
|
|
xcat-core packages are Perl, so one build serves every Ubuntu release: the
|
|
packages are built **once** and the same files are published into every codename
|
|
the repository declares. Only ``xCAT``, ``xCATsn`` and ``xCAT-genesis-scripts``
|
|
carry an architecture, and there the difference is packaging metadata rather than
|
|
compiled output. That is why this build needs no ``sbuild`` and no per-codename
|
|
chroot -- unlike xcat-deps, whose packages are compiled and genuinely differ per
|
|
release.
|
|
|
|
Helpers shared by both builders live in ``BuildUtils.pm``.
|
|
|
|
``buildcore.sh`` builds the architecture specific packages (``xCAT``, ``xCATsn``,
|
|
``xCAT-genesis-scripts``) for every supported architecture, riscv64 included, with
|
|
``rpmbuild --target``. The build host therefore needs an rpm that knows the riscv64
|
|
architecture and the ``Recommends:`` tag (EL8 or later); an older host fails the riscv64
|
|
builds and, as the script only publishes when every architecture built, produces no rpms.
|
|
|
|
xcat-deps
|
|
---------
|
|
|
|
The ``xcat-deps`` package is currently owned and maintained by the core development on our internal servers. Use the packages created at: http://xcat.org/download.html#xcat-dep
|
|
|
|
|
|
man pages
|
|
---------
|
|
|
|
The xCAT man pages are written in Perl POD files and automatically get built into the xCAT rpms. The content in the .pod files are always the master.
|
|
|
|
In the past, the man pages were converted into html files and uploaded to SourceForge. In moving to `ReadTheDocs <http://xcat-docs.readthedocs.org>`_ we want to also provide the man pages as references in the documentation. To convert the ``pods`` to ``rst``, we are using The Perl module: `pod2rst <http://search.cpan.org/~dowens/Pod-POM-View-Restructured-0.02/bin/pod2rst>`_.
|
|
|
|
The following steps will help configure ``pod2rst`` and be able to generate the changes .rst files to push to GitHub.
|
|
|
|
#. Download the following Perl modules:
|
|
|
|
- `Pod-POM-View-Restructured-0.02 <http://search.cpan.org/~dowens/Pod-POM-View-Restructured-0.02/lib/Pod/POM/View/Restructured.pm>`_
|
|
- `Pod-POM-2.00 <http://search.cpan.org/~neilb/Pod-POM-2.00/lib/Pod/POM.pm>`_
|
|
|
|
#. For each of the above Perl modules:
|
|
|
|
* **[as root]** Extract and build the Perl module ::
|
|
|
|
perl Makefile.PL
|
|
make
|
|
make install
|
|
|
|
* **[as non-root]** Extract and build the Perl module using PREFIX to specify a directory that you have write permission ::
|
|
|
|
mkdir ~/perllib
|
|
perl Makefile.PL PREFIX=~/perllib
|
|
make
|
|
make install
|
|
|
|
#. Execute the script ``create_man_pages.py`` to generate the .rst files into ``xcat-core/docs`` :
|
|
|
|
* **[as root]** ::
|
|
|
|
cd xcat-core
|
|
./create_man_pages.py
|
|
|
|
* **[as non root]** ::
|
|
|
|
cd xcat-core
|
|
./create_man_pages.py --prefix=~/perllib
|