mirror of
https://github.com/xcat2/xcat-core.git
synced 2025-06-13 09:50:19 +00:00
Merge pull request #858 from immarvin/onxcatindocker
refine the "run xcat in docker" doc, add doc for starting xCAT container with compose
This commit is contained in:
Xiaopeng Wang
@ -0,0 +1,97 @@
|
||||
Dockerized xCAT
|
||||
================
|
||||
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
`Docker <https://www.docker.com/>`_ is a popular containment environment making it easier to create, deploy, and run applications/services. With Docker, applications/services are shipped as **Docker images** and run in **Docker containers**. **Docker containers** include the application/services and all of its running environment, which have isolated namespace and network stack but share the kernel on the host operating system. The server on which **Docker containers** run is called **Docker host**. **Docker image** can only run on **Docker host** with the same architecture.
|
||||
|
||||
|
||||
.. image:: ./imgs/xcatindocker4.jpg
|
||||
:height: 500 px
|
||||
:width: 600 px
|
||||
:scale: 50 %
|
||||
:alt: alternate text
|
||||
:align: right
|
||||
|
||||
|
||||
**Compose** is a native tool shipped by Docker to define and run applications in Docker containers. You use a Compose file to configure your applications/services. Then, using a single command, you create and start all the services from your configuration.
|
||||
|
||||
By pulling xCAT Docker image and running xCAT Docker image in a container, you get a well-configured xCAT Management Node to start cluster management work, without worrying about the xCAT installation and configuration on different OS and various hardware platforms.
|
||||
|
||||
|
||||
xCAT Docker images
|
||||
------------------
|
||||
|
||||
xCAT shippes 2 Docker images for Docker host with different architecture:
|
||||
|
||||
* "xcat/xcat-ubuntu-x86_64": run on x86_64 Docker host
|
||||
* "xcat/xcat-ubuntu-ppc64le": run on ppc64le Docker host
|
||||
|
||||
Each of the xCAT Docker images above has 3 tags corresponding to different xCAT release inside Docker image:
|
||||
|
||||
* "latest" : the latest xCAT release
|
||||
* "2.11" : xCAT 2.11 release
|
||||
* "dev" : the latest xCAT development snapshot build
|
||||
|
||||
A Docker image with specified tag should be denoted in format "[docker image name]:[tag]", for example, "xcat/xcat-ubuntu-x86_64:2.11". If not specified, default tag is "latest".
|
||||
|
||||
|
||||
Run xCAT in Docker
|
||||
------------------
|
||||
|
||||
Each container with xCAT Docker image running inside is a xCAT management node, the container connects to the compute nodes and hardware control points in the cluster via "bridge" network on the Docker host. Generally, a xCAT container should connect to 2 types of networks( the 2 types of networks might be one network in some cluster):
|
||||
|
||||
* "mgtnet": Management network, the network used by the Management Node to install operating systems and manage the nodes. The Management Node and in-band Network Interface Card (NIC) of the nodes are connected to this network. A bridge "mgtbr" will be created and attached to the network interface facing the compute nodes on Docker host
|
||||
* "svcnet": Service network, the network used by the Management Node to control the nodes using out-of-band management using the Service Processor. A bridge "svcbr" will be created and attached to the network interface facing the hardware control points
|
||||
|
||||
You are required to determine and specify some necessary information, so that xCAT is well configured and running when the container is started. This includes:
|
||||
|
||||
* Docker container: xCAT Docker image with specified xCAT release; the data volumes with directories on Docker host including xCAT DB tables, the osimage resources and xCAT logs, which can be used to save and restore the data of xCAT service
|
||||
* network information: the network configuration of the xCAT container
|
||||
* cluster information: the domain of the cluster
|
||||
|
||||
The information can be specified in 2 ways to run xCAT container:
|
||||
|
||||
* in options and arguments of docker commands such as ``docker network create`` or ``docker run``
|
||||
* in the "docker-compose.yml", which contains all the configuration to start xCAT containers with Compose. This is the recommended way to start xCAT container.
|
||||
|
||||
The 2 ways are described step-by-step in the following sections of this documentation. We recommend you start xCAT container with Compose.
|
||||
|
||||
When xCAT Docker container is started, you can access it with ``sudo docker attach``, however, we recommend you to access it via "ssh".
|
||||
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
setup_docker_host.rst
|
||||
run_xcat_in_docker_compose.rst
|
||||
run_xcat_in_docker_native.rst
|
||||
|
||||
|
||||
|
||||
Work with xCAT
|
||||
--------------
|
||||
|
||||
Once xCAT Docker container is running, you can use xCAT with the shell inside the container. Since the ssh service has been enabled on the Docker container startup, you can connect to the container via ssh. The default root password is "cluster".
|
||||
|
||||
Once you attach or ssh to the container, you will find that xCAT is running and configured, you can play with xCAT and manage your cluster now.
|
||||
|
||||
Currently, since xCAT can only generate the diskless osimages of Linux distributions with the same OS version and architecture with xCAT MN. If you need to provision diskless osimages besides ubuntu x86_64 with xCAT running in the Docker, you can use ``imgexport`` and ``imgimport`` to import the diskless osimages generated before.
|
||||
|
||||
If you start up the xCAT Docker container by following the steps described in sections above strictly, without specifying "--dns=IP_ADDRESS...", "--dns-search=DOMAIN...", or "--dns-opt=OPTION..." options, Docker uses the /etc/resolv.conf of the host machine (where the docker daemon runs). Any DNS problem inside container, please make sure the DNS server on the Docker host works well.
|
||||
|
||||
Save and Restore xCAT data
|
||||
----------------------------
|
||||
|
||||
According to the policy of Docker, Docker image should only be the service deployment unit, it is not recommended to save data in Docker image. Docker uses "Data Volume" to save persisent data inside container, which can be simply taken as a shared directory between Docker host and Docker container.
|
||||
|
||||
For dockerized xCAT, there are 3 volumes recommended to save and restore xCAT user data.
|
||||
|
||||
* "/install": save the osimage resources under "/install" directory
|
||||
* "/var/log/xcat/": save xCAT logs
|
||||
* "/.dbbackup": save and restore xCAT DB tables. You can save the xCAT DB tables with ``dumpxCATdb -p /.dbbackup/`` inside container and xCAT will restore the tables on the container start up.
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 87 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 90 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 81 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 93 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 85 KiB |
@ -0,0 +1,154 @@
|
||||
Run xCAT in Docker with Compose (Recommended)
|
||||
=============================================
|
||||
|
||||
|
||||
An example configuration in the documentation
|
||||
---------------------------------------------
|
||||
|
||||
To demonstrate the steps to run xCAT in a Docker container, take a cluster with the following configuration as an example ::
|
||||
|
||||
|
||||
The name of the docker container running xCAT: xcatmn
|
||||
The hostname of container xcatmn: xcatmn
|
||||
The dns domain of the cluster: clusters.com
|
||||
|
||||
The management network object: mgtnet
|
||||
The network bridge of management network on Docker host: mgtbr
|
||||
The management network interface on the Docker host facing the compute nodes: eno1
|
||||
The IP address of eno1 on Docker host: 10.5.107.1/8
|
||||
The IP address of xCAT container in management network: 10.5.107.101
|
||||
|
||||
The service network object: svcnet
|
||||
The network bridge of service network on Docker host: svcbr
|
||||
The service network interface on the Docker host facing the hardware control points: eno2
|
||||
The IP address of eno2 on Docker host: 192.168.0.1/8
|
||||
The IP address of xCAT container in service network: 192.168.0.101
|
||||
|
||||
|
||||
Install Compose on Docker host
|
||||
------------------------------
|
||||
|
||||
Compose v1.7.0 or above should be installed on Docker host: ::
|
||||
|
||||
curl -L https://github.com/docker/compose/releases/download/1.7.0-rc1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
|
||||
chmod +x /usr/local/bin/docker-compose
|
||||
|
||||
|
||||
Customize docker-compose file
|
||||
-----------------------------
|
||||
|
||||
xCAT ships a docker-compose template `docker-compose.yml <https://github.com/immarvin/xcat-docker/blob/master/docker-compose.yml>`_, which is a self-description file including all the configurations to run xCAT in container. You can make up your compose file based on it if you are familiar with `Compose file <https://docs.docker.com/compose/compose-file/>`_ , otherwise, you can simply customize it with the following steps:
|
||||
|
||||
1. Specify the xCAT Docker image
|
||||
|
||||
::
|
||||
|
||||
image: [xCAT Docker image name]:[tag]
|
||||
|
||||
specify the name and tag of xCAT Docker image, for example "xcat/xcat-ubuntu-x86_64:2.11"
|
||||
|
||||
2. Specify the cluster domain name
|
||||
|
||||
::
|
||||
|
||||
extra_hosts:
|
||||
- "xcatmn.[cluster domain name] xcatmn:[Container's IP address in management network]"
|
||||
|
||||
specify the cluster domain name,i.e, "site.domain" on xCAT Management Node, for example "clusters.com", and the IP address of xCAT Docker container in the management network, such as "10.5.107.101"
|
||||
|
||||
3. Specify the IP address of xCAT container in service network and management network
|
||||
|
||||
::
|
||||
|
||||
networks:
|
||||
|
||||
svcnet:
|
||||
ipv4_address : [Container's IP address in service network]
|
||||
|
||||
mgtnet:
|
||||
ipv4_address : [Container's IP address in management network]
|
||||
|
||||
specify the IP address of Docker container in service network and management network. If the "svcnet" is the same as "mgtnet", the 2 "svcnet" lines should be commented out.
|
||||
|
||||
4. Specify the Docker network objects for management network and service network
|
||||
|
||||
::
|
||||
|
||||
networks:
|
||||
|
||||
#management network, attached to the network interface on Docker host
|
||||
#facing the nodes to provision
|
||||
mgtnet:
|
||||
driver: "bridge"
|
||||
driver_opts:
|
||||
com.docker.network.bridge.name: "mgtbr"
|
||||
ipam:
|
||||
config:
|
||||
- subnet: [subnet of mgtbr in CIDR]
|
||||
gateway:[IP address of mgtbr]
|
||||
|
||||
#service network, attached to the network interface on
|
||||
#Docker host facing the bmc network
|
||||
svcnet:
|
||||
driver: "bridge"
|
||||
driver_opts:
|
||||
com.docker.network.bridge.name: "svcbr"
|
||||
ipam:
|
||||
config:
|
||||
- subnet: [subnet of svcbr in CIDR]
|
||||
gateway: [IP address of svcbr]
|
||||
|
||||
specify the network configuration of bridge networks "mgtnet" and "svcnet", the network configuration of the bridge networks should be same as the network interfaces attached to the bridges. The "mgtnet" and "svcnet" might the same network in some cluster, in this case, you can ignore the lines for "svcnet".
|
||||
|
||||
5. Specify the Data Volumes for xCAT Docker container
|
||||
|
||||
::
|
||||
|
||||
volumes:
|
||||
#the "/install" volume is used to keep user data in xCAT,
|
||||
#such as osimage resources
|
||||
#the user data can be accessible if specified
|
||||
- [The directory on Docker host mounted to "/install" inside container]:/install
|
||||
#the "/.dbbackup" volume is used to backup and restore xCAT DB tables
|
||||
#Dockerized xCAT will restore xCAT DB tables if specified
|
||||
#"dumpxCATdb -p /.dbbackup" should be run manually to save xCAT DB inside container
|
||||
- [The directory on Docker host mounted to save xCAT DB inside container]:/.dbbackup
|
||||
#the "/.logs" value is used to keep xCAT logs
|
||||
#the xCAT logs will be kept if specified
|
||||
- [The directory on Docker host to save xCAT logs inside container]:/var/log/xcat/
|
||||
|
||||
specify the volumes of the xCAT container used to save and restore xCAT data
|
||||
|
||||
|
||||
Start xCAT Docker container with Compose
|
||||
----------------------------------------
|
||||
After the "docker-compose.yml" is ready, the xCAT Docker container can be started with [1]_ ::
|
||||
|
||||
docker-compose -f "docker-compose.yml" up -d; \
|
||||
ifconfig eno1 0.0.0.0; \
|
||||
brctl addif mgtbr eno1; \
|
||||
ip link set mgtbr up; \
|
||||
docker-compose logs -f
|
||||
|
||||
This command starts up the Docker container and attaches the network interface "eno1" of Docker host to the bridge network "mgtbr". It is a little complex due to a Compose bug `#1003 <https://github.com/docker/libnetwork/issues/1003>`_ . The commands should be run successively in one line to avoid breaking the network connection of the network interface of Docker host.
|
||||
|
||||
To remove the container, you can run ::
|
||||
|
||||
docker-compose -f "docker-compose.yml" down; \
|
||||
ifdown eno1; \
|
||||
ifup eno1
|
||||
|
||||
To update the xCAT Docker image, you can run ::
|
||||
|
||||
docker-compose -f "docker-compose.yml" pull
|
||||
|
||||
|
||||
Known Issues
|
||||
------------
|
||||
|
||||
.. [1] When you start up xCAT Docker container, you might see an error message at the end of the output like: ::
|
||||
|
||||
"Couldn't connect to Docker daemon at http+unix://var/run/docker.sock - is it running? If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable."
|
||||
|
||||
You can ignore it, the container has already been running. It is a Docker bug `#1214 <https://github.com/docker/compose/issues/1214>`_
|
||||
|
||||
@ -0,0 +1,77 @@
|
||||
Run xCAT in Docker with Docker native commands
|
||||
==============================================
|
||||
|
||||
|
||||
Pull the xCAT Docker image from DockerHub
|
||||
-----------------------------------------
|
||||
|
||||
Now xCAT ships xCAT Docker images(x86_64 and ppc64le) on the `DockerHub <https://hub.docker.com/u/xcat/>`_:
|
||||
|
||||
To pull the latest xCAT Docker image for x86_64, run ::
|
||||
|
||||
sudo docker pull xcat/xcat-ubuntu-x86_64
|
||||
|
||||
On success, you will see the pulled Docker image on Docker host ::
|
||||
|
||||
[root@dockerhost1 ~]# sudo docker images
|
||||
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
|
||||
xcat/xcat-ubuntu-x86_64 latest 3a3631463e83 2 days ago 643 MB
|
||||
|
||||
|
||||
An example configuration in the documentation
|
||||
---------------------------------------------
|
||||
|
||||
To demonstrate the steps to run xCAT in a Docker container, take a cluster with the following configuration as an example ::
|
||||
|
||||
Docker host: dockerhost1
|
||||
The name of the docker container running xCAT: xcatmn
|
||||
The hostname of container xcatmn: xcatmn
|
||||
|
||||
The management network object: mgtnet
|
||||
The network bridge of management network on Docker host: mgtbr
|
||||
The management network interface on the Docker host facing the compute nodes: eno1
|
||||
The IP address of eno1 on Docker host: 10.5.107.1/8
|
||||
The IP address of xCAT container in management network: 10.5.107.101
|
||||
|
||||
The dns domain of the cluster: clusters.com
|
||||
|
||||
|
||||
Create a customized Docker network on the Docker host
|
||||
-----------------------------------------------------
|
||||
|
||||
**Docker Networks** provide complete isolation for containers, which gives you control over the networks your containers run on. To run xCAT in Docker, you should create a customized bridge network according to the cluster network plan, instead of using the default bridge network created on Docker installation.
|
||||
|
||||
As an example, we create a customized bridge network "mgtbr" which is attached to the network interface "eno1" facing the compute nodes and inherits the network configuration of "eno1". Since the commands to create the network will break the network connection on "eno1", you'd better run the commands in one line instead of running them seperatly ::
|
||||
|
||||
sudo docker network create --driver=bridge --gateway=10.5.107.1 --subnet=10.5.107.0/8 -o "com.docker.network.bridge.name"="mgtbr" mgtnet; \
|
||||
ifconfig eno1 0.0.0.0; \
|
||||
brctl addif mgtbr eno1; \
|
||||
ip link set mgtbr up
|
||||
|
||||
* ``--driver=bridge`` specify the network driver to be "bridge"
|
||||
* ``--gateway=10.5.107.1`` specify the network gateway to be the IP address of "eno1" on Docker host. which will also be the IP address of network bridge "mgtbr"
|
||||
* ``--subnet=10.5.107.0/8`` speify the subnet in CIDR format to be the subnet of "eno1"
|
||||
* ``com.docker.network.bridge.name"="mgtbr"`` specify the bridge name of management network
|
||||
* ``ifconfig eno1 0.0.0.0`` delete the IP address of "eno1"
|
||||
* ``brctl addif mgtbr eno1`` attach the bridge "br0" to network interface "eno1"
|
||||
* ``ip link set mgtbr up`` change the state of "br0" to UP
|
||||
|
||||
When the network is created, you can list it with ``sudo docker network ls`` and get the information of it with ``sudo docker inspect mgtnet``.
|
||||
|
||||
|
||||
Run xCAT in Docker container
|
||||
----------------------------
|
||||
|
||||
Now run the xCAT Docker container with the Docker image "xcat/xcat-ubuntu-x86_64" and connect it to the newly created customized Docker network "mgtnet" ::
|
||||
|
||||
sudo docker run -it --privileged=true --hostname=xcatmn --name=xcatmn --add-host="xcatmn.clusers.com xcatmn:10.5.107.101" --volume /docker/xcatdata/:/install --net=mgtnet --ip=10.5.107.101 xcat/xcat-ubuntu-x86_64
|
||||
|
||||
* use ``--privileged=true`` to give extended privileges to this container
|
||||
* use ``--hostname`` to specify the hostname of the container, which is available inside the container
|
||||
* use ``--name`` to assign a name to the container, this name can be used to manipulate the container on Docker host
|
||||
* use ``--add-host="xcatmn.clusers.com xcatmn:10.5.107.101"`` to write the ``/etc/hosts`` entries of Docker container inside container. Since xCAT use the FQDN(Fully Qualified Domain Name) to determine the cluster domain on startup, please make sure the format to be "<FQDN> <hostname>: <IP Address>", otherwise, you need to set the cluster domain with ``chdef -t site -o clustersite domain="clusters.com"`` inside the container manually
|
||||
* use ``--volume /docker/xcatdata/:/install`` to mount a pre-created "/docker/xcatdata" directory on Docker host to "/install" directory inside container as a data volume. This is optional, it is mandatory if you want to backup and restore xCAT data.
|
||||
* use ``--net=mgtnet`` to connect the container to the Docker network "mgtnet"
|
||||
* use ``--ip=10.5.107.101`` to specify the IP address of the xCAT Docker container
|
||||
|
||||
|
||||
@ -0,0 +1,28 @@
|
||||
Setup Docker host
|
||||
=================
|
||||
|
||||
Install Docker Engine
|
||||
---------------------
|
||||
|
||||
The Docker host to run xCAT Docker image should be a baremental or virtual server with Docker v1.10 or above installed. For the details on system requirements and Docker installation, please refer to `Docker Installation Docs <https://docs.docker.com/engine/installation/>`_.
|
||||
|
||||
**Note:**
|
||||
|
||||
1. **Docker image** can only run on **Docker host** with the same architecture. Since xCAT currently only ships x86_64 and ppc64le Docker images, running xCAT in Docker requires x86_64 or ppc64le **Docker hosts**.
|
||||
|
||||
|
||||
Shutdown the SELinux/Apparmor on Docker host
|
||||
--------------------------------------------
|
||||
|
||||
If the SELinux or Apparmor on Docker host is enabled, the services/applications inside Docker Container might be confined. To run xCAT in Docker container, SELinux and Apparmor on the Docker host must be disabled.
|
||||
|
||||
SELinux can be disabled with: ::
|
||||
|
||||
echo 0 > /selinux/enforce
|
||||
sed -i 's/^SELINUX=.*$/SELINUX=disabled/' /etc/selinux/config
|
||||
|
||||
AppArmor can be disabled with: ::
|
||||
|
||||
/etc/init.d/apparmor teardown
|
||||
|
||||
|
||||
@ -4,6 +4,6 @@ Docker
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
run_xcat_in_docker.rst
|
||||
dockerized_xcat/dockerized_xcat.rst
|
||||
lifecycle_management.rst
|
||||
docker_registry.rst
|
||||
|
||||
@ -1,135 +0,0 @@
|
||||
Run xCAT in Docker Container
|
||||
============================
|
||||
|
||||
`Docker <https://www.docker.com/>`_ is a popular application containment environment. With Docker, applications/Services are shipped as **Docker images** and run in **Docker containers**. **Docker containers** include the application and all of its dependencies, but share the kernel with other containers. They run as an isolated process in userspace on the host operating system. The server on which **Docker containers** run is called **Docker host**.
|
||||
|
||||
When running xCAT in Docker container, you do not have to worry about the xCAT installation and configuration on different OS and hardware platforms, just focus on the cluster management work with xCAT.
|
||||
|
||||
|
||||
Prerequisite: setup Docker host
|
||||
--------------------------------
|
||||
|
||||
You can select a baremental or virtual server with the Docker installed as a Docker host. For the details on system requirements and Docker installation, please refer to `Docker Docs <https://docs.docker.com/>`_.
|
||||
|
||||
**Note:**
|
||||
|
||||
1. **Docker image** can only run on the **Docker host** with the same architecture. Since xCAT currently only ships x86_64 and ppc64le Docker images, running xCAT in Docker requires x86_64 or ppc64le **Docker hosts**.
|
||||
|
||||
2. **Docker v1.10** introduces significant enhancements and changes from previous releases, please make sure the Docker release installed on Docker host is newer than Docker v1.10.
|
||||
|
||||
|
||||
Shutdown the SELinux/Apparmor on Docker host
|
||||
--------------------------------------------
|
||||
|
||||
If the SELinux or Apparmor on Docker host is enabled, the services/applications inside Docker Container might be confined. To run xCAT in Docker container, SELinux and Apparmor on the Docker host must be disabled.
|
||||
|
||||
SELinux can be disabled with: ::
|
||||
|
||||
echo 0 > /selinux/enforce
|
||||
sed -i 's/^SELINUX=.*$/SELINUX=disabled/' /etc/selinux/config
|
||||
|
||||
AppArmor can be disabled with: ::
|
||||
|
||||
/etc/init.d/apparmor teardown
|
||||
|
||||
|
||||
Pull the xCAT Docker image from DockerHub:
|
||||
------------------------------------------
|
||||
|
||||
Now xCAT ships xCAT 2.11 Docker images(x86_64 and ppc64le) on the `DockerHub <https://hub.docker.com/u/xcat/>`_:
|
||||
|
||||
To pull the xCAT 2.11 Docker for x86_64, run ::
|
||||
|
||||
[root@dockerhost1 ~]# sudo docker pull xcat/xcat-ubuntu-x86_64
|
||||
Using default tag: latest
|
||||
latest: Pulling from xcat/xcat-ubuntu-x86_64
|
||||
118aadd1f859: Already exists
|
||||
41402770caf2: Already exists
|
||||
a5051dd98acd: Already exists
|
||||
a3ed95caeb02: Already exists
|
||||
b084cef63fa6: Already exists
|
||||
f993e0b41814: Already exists
|
||||
70da11abb463: Already exists
|
||||
ef43498c5fbc: Already exists
|
||||
Digest: sha256:1dd0b80d4ff91ed9ddd11a3f16c10d33553cf2acf358f72575d9290596a89157
|
||||
Status: Image is up to date for xcat/xcat-ubuntu-x86_64:latest
|
||||
|
||||
On success, you will see the pulled Docker image on Docker host ::
|
||||
|
||||
[root@dockerhost1 ~]# sudo docker images
|
||||
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
|
||||
xcat/xcat-ubuntu-x86_64 latest 3a3631463e83 2 days ago 643 MB
|
||||
|
||||
|
||||
An example configuration in the documentation
|
||||
---------------------------------------------
|
||||
|
||||
To demonstrate the steps to run xCAT in a Docker container, take a cluster with the following configuration as an example ::
|
||||
|
||||
Docker host: dockerhost1
|
||||
The network interface on the Docker host facing the compute nodes: eno1
|
||||
The IP address of eno1 on Docker host: 10.5.107.1/8
|
||||
The customized docker bridge: br0
|
||||
The name of the docker container running xCAT: xcatmn
|
||||
The hostname of container xcatmn: xcatmn
|
||||
The IP address of container xcatmn: 10.5.107.101
|
||||
The dns domain of the cluster: clusters.com
|
||||
|
||||
|
||||
Create a customized Docker network on the Docker host
|
||||
-----------------------------------------------------
|
||||
|
||||
**Docker Networks** provide complete isolation for containers, which gives you control over the networks your containers run on. To run xCAT in Docker, you should create a customized bridge network according to the cluster network plan, instead of using the default bridge network created on Docker installation.
|
||||
|
||||
As an example, we create a customized bridge network "subnet1" which is attached to the network interface "eno1" facing the compute nodes and inherits the network configuration of "eno1". Since the commands to create the network will break the network connection on "eno1", you'd better run the commands in one line instead of running them seperatly ::
|
||||
|
||||
[root@dockerhost1 ~]# sudo docker network create --driver=bridge --gateway=10.5.107.1 --subnet=10.5.107.0/8 --ip-range=10.5.107.100/30 -o "com.docker.network.bridge.name"="br0" -o "com.docker.network.bridge.host_binding_ipv4"="10.5.107.1" subnet1;ip addr del dev eno1 10.5.107.1/8;brctl addif br0 eno1;ip link set br0 up
|
||||
|
||||
* ``--driver=bridge`` specify the network driver to be "bridge"
|
||||
* ``--gateway=10.5.107.1`` specify the network gateway to be the IP address of "eno1" on Docker host
|
||||
* ``--subnet=10.5.107.0/8`` speify the subnet in CIDR format to be the subnet of "eno1"
|
||||
* ``--ip-range=10.5.107.100/30`` specify the sub-range to allocate container IP, this should be a segment of subnet specified with "--subnet"
|
||||
* ``-o "com.docker.network.bridge.name"="br0" -o "com.docker.network.bridge.host_binding_ipv4"="10.5.107.1"`` specify the specific options for "bridge" driver. ``com.docker.network.bridge.name"="br0"`` specify the name of the bridge created to be "br0", ``"com.docker.network.bridge.host_binding_ipv4"="10.5.107.1"`` specify the IP address of the bridge "br0", which is the IP address of the network interface "eno1"
|
||||
* ``ip addr del dev eno1 10.5.107.1/8`` delete the IP address of "eno1"
|
||||
* ``brctl addif br0 eno1`` attach the bridge "br0" to network interface "eno1"
|
||||
* ``ip link set br0 up`` change the state of "br0" to UP
|
||||
|
||||
When the network is created, you can list it with ``sudo docker network ls`` and get the information of it with ``sudo docker inspect subnet1``.
|
||||
|
||||
|
||||
Run xCAT in Docker container
|
||||
----------------------------
|
||||
|
||||
Now run the xCAT Docker container with the Docker image "xcat/xcat-ubuntu-x86_64" and connect it to the newly created customized Docker network "subnet1" ::
|
||||
|
||||
[root@dockerhost1 ~]# sudo docker run -it --privileged=true --hostname=xcatmn --name=xcatmn --add-host="xcatmn.clusers.com xcatmn:10.5.107.101" --volume /docker/xcatdata/:/install --net=subnet1 --ip=10.5.107.101 xcat/xcat-ubuntu-x86_64
|
||||
|
||||
* use ``--privileged=true`` to give extended privileges to this container
|
||||
* use ``--hostname`` to specify the hostname of the container, which is available inside the container
|
||||
* use ``--name`` to assign a name to the container, this name can be used to manipulate the container on Docker host
|
||||
* use ``--add-host="xcatmn.clusers.com xcatmn:10.5.107.101"`` to write the ``/etc/hosts`` entries of Docker container inside container. Since xCAT use the FQDN(Fully Qualified Domain Name) to determine the cluster domain on startup, please make sure the format to be "<FQDN> <hostname>: <IP Address>", otherwise, you need to set the cluster domain with ``chdef -t site -o clustersite domain="clusters.com"`` inside the container manually
|
||||
* use ``--volume /docker/xcatdata/:/install`` to mount a pre-created "/docker/xcatdata" directory on Docker host to "/install" directory inside container as a data volume. This is optional, it is mandatory if you want to backup and restore xCAT data.
|
||||
* use ``--net=subnet1`` to connect the container to the Docker network "subnet1"
|
||||
* use ``--ip=10.5.107.101`` to specify the IP address of the Docker container
|
||||
|
||||
|
||||
Play with xCAT
|
||||
--------------
|
||||
|
||||
Once xCAT Docker container is run, you can use xCAT with the shell inside the container. Since the ssh service has also been enabled on the Docker container startup, you can also connect to the container via ssh, the default password for the user "root" is "cluster".
|
||||
|
||||
Once you attach or ssh to the container, you will find that xCAT is running and configured, you can play with xCAT and manage your cluster now.
|
||||
|
||||
Currently, since xCAT can only generate the diskless osimages of Linux distributions with the same OS version and architecture with xCAT MN. If you need to provision diskless osimages besides ubuntu x86_64 with xCAT running in the Docker, you can use ``imgexport`` and ``imgimport`` to import the diskless osimages generated before.
|
||||
|
||||
Save and Restore xCAT data
|
||||
----------------------------
|
||||
|
||||
It is not recommended to save data in Docker image. "/install" directory inside Docker container is the right place to backup xCAT DB tables, save osimage resource files and other user data.
|
||||
|
||||
You can specify a directory on the Docker host as a data volume for the "/install" directory inside container. xCAT will preserve several directories under "/install" for special use:
|
||||
|
||||
* save the osimage resources under "/install"
|
||||
* save xCAT logs under "/install/.logs" directory
|
||||
* create a directory "/install/.dbbackup" as the place to save and restore xCAT DB tables. You can save the xCAT DB tables with ``dumpxCATdb -p /install/.dbbackup/`` and xCAT will restore the tables on the container start up.
|
||||
|
||||
Reference in New Issue
Block a user