diff --git a/docs/source/advanced/docker/lifecycle_management.rst b/docs/source/advanced/docker/lifecycle_management.rst index cd3385d20..100c71969 100644 --- a/docs/source/advanced/docker/lifecycle_management.rst +++ b/docs/source/advanced/docker/lifecycle_management.rst @@ -5,7 +5,8 @@ The Docker linux container technology is currently very popular. xCAT can help m This document describes how to use xCAT for docker management, from Docker Host setup to docker container operationis. -**Note:** The document is based on **Docker Version 1.10.x** and **Docker API version 1.22.** And the Docker Host is based on **ubuntu14.04.3 x86_64**. At the time of this writing (February 2016), docker host images are not available for **ppc64** architecture from docker.org. You can search online to find them or build your own. +**Note:** The document was verified with **Docker Version 1.10, 1.11** and **Docker API version 1.22.** The Docker Host was verified on **ubuntu14.04.3 x86_64**, **ubuntu15.10 x86_64**, **ubuntu16.04 x86_64** and **ubuntu16.04 ppc64el**. + Setting up Docker Host ---------------------- @@ -23,17 +24,20 @@ The osimage represents the image of the Operating System which will be deployed Copy files out from DVDs/ISOs and generate """""""""""""""""""""""""""""""""""""""""" - -:: +**[ubuntu x86_64]** :: - copycds ubuntu-14.04.3-server-amd64.iso + copycds ubuntu-xxx-server-amd64.iso + +**[ubuntu16.04 ppc64el]** :: + + copycds ubuntu-16.04-server-ppc64el.iso Create pkglist and otherpkglist of osimage for dockerhost """"""""""""""""""""""""""""""""""""""""""""""""""""""""" The pkglist file should contain the following: :: - # cat /install/custom/ubuntu1404/ubuntu1404.pkglist + # cat /install/custom/ubuntu/ubuntu.pkglist openssh-server ntp gawk @@ -41,14 +45,38 @@ The pkglist file should contain the following: :: snmpd bridge-utils -The otherpkglist file should contain the following: :: +The otherpkglist file should contain the following: - # cat /install/custom/ubuntu1404/ubuntu1404_docker.pkglist +**[ubuntu x86_64]** :: + + # cat /install/custom/ubuntu/ubuntu_docker.pkglist docker-engine +**[ubuntu16.04 ppc64el]** + +At the time of this writing (February 2016), docker package is not available for **ppc64el** architecture from docker.org. You can follow instructions below on how to manually download and install it. + +* Download docker engine for ppc64el +:: + + wget http://launchpadlibrarian.net/251622081/docker.io_1.10.3-0ubuntu4_ppc64el.deb -O /install/docker_ppc64el/docker.io_1.10.3-0ubuntu4_ppc64el.deb + +* Configure **otherpkgdir** like this +:: + + otherpkgdir=/install/docker_ppc64el + +* The **otherpkglist** file should be +:: + + # cat /install/custom/ubuntu/ubuntu_docker.pkglist + docker.io + Create the osimage for dockerhost """"""""""""""""""""""""""""""""" -The osimage for dockerhost will be like this: :: +The osimage for dockerhost will be like this: + +**[ubuntu x86_64]** :: # lsdef -t osimage ub14.04.03-x86_64-dockerhost Object name: ub14.04.03-x86_64-dockerhost @@ -57,13 +85,29 @@ The osimage for dockerhost will be like this: :: osname=Linux osvers=ubuntu14.04.3 otherpkgdir=https://apt.dockerproject.org/repo ubuntu-trusty main,http://cz.archive.ubuntu.com/ubuntu trusty main - otherpkglist=/install/custom/ubuntu1404/ubuntu1404_docker.pkglist + otherpkglist=/install/custom/ubuntu/ubuntu_docker.pkglist pkgdir=/install/ubuntu14.04.3/x86_64 - pkglist=/install/custom/ubuntu1404/ubuntu1404.pkglist + pkglist=/install/custom/ubuntu/ubuntu.pkglist profile=compute provmethod=install template=/opt/xcat/share/xcat/install/ubuntu/compute.tmpl +**[ubuntu16.04 ppc64el]** :: + + # lsdef -t osimage ub16.04-ppc64el-dockerhost + Object name: ub16.04-ppc64el-dockerhost + imagetype=linux + osarch=ppc64el + osname=Linux + osvers=ubuntu16.04 + otherpkgdir=/install/docker_ppc64el + otherpkglist=/install/custom/ubuntu/ubuntu_docker.pkglist + pkgdir=/install/ubuntu16.04/ppc64el + pkglist=/install/custom/ubuntu/ubuntu.pkglist + profile=compute + provmethod=install + template=/opt/xcat/share/xcat/install/ubuntu/compute.tmpl + Preparing setup trust connection for docker service and create docker network object ```````````````````````````````````````````````````````````````````````````````````` Currently, a customer defined network object is needed when create a docker container with static IP address, it can be done with the command: :: @@ -99,7 +143,7 @@ After the dockerhost is ready, a docker instance can be managed through xCAT com postbootscripts=otherpkgs postscripts=syslog,remoteshell,syncfiles -The command :doc:`mkdef ` or :doc:`chdef ` can be used to create a new docker instance node or change the node attributes. Specify any available unused ip address for *ip* attribute. +The command :doc:`mkdef ` or :doc:`chdef ` can be used to create a new docker instance node or change the node attributes. Specify any available unused ip address for *ip* attribute. *mac* attribute is optional and if left unset, will be filled in by *mkdocker* command. After docker instance node is defined, use command `makehosts host01c01` to add node *host01c01* and its IP address *10.0.120.1* into /etc/hosts. @@ -173,3 +217,26 @@ Check docker instance status :: rpower state + +Troubleshooting +-------------------------- + +If things go wrong: + +* After dockerhost node boots, check contents of **/var/log/xcat/xcat.log** file on the dockerhost for errors. + +* Verify **nicname** specified in **Preparing setup trust connection for docker service and create docker network object** section exists on the docker host. Depending on the version of Ubuntu OS and host architecture, it could be **eth0**, or **em1**, or **eno1**, or **enp0s1**. Verify by running on the dockerhost +:: + + ip addr show dev + +* Run **ps -ef | grep docker** to verify docker engine is running with configured options. It should look something like +:: + + root 3703 1 0 Apr15 ? 00:12:28 /usr/bin/docker daemon -H unix:///var/run/docker.sock -H tcp://host01:2375 --tls --tlscacert=/root/.docker/ca-cert.pem --tlscert=/root/.docker/dockerhost-cert.pem --tlskey=/root/.docker/dockerhost-cert.pem --tlsverify=true --raw-logs + +If the output is missing some options, verify that file **/lib/systemd/system/docker.service** contains the following lines +:: + + EnvironmentFile=-/etc/default/docker + ExecStart=/usr/bin/docker daemon $DOCKER_OPTS -H fd:// diff --git a/xCAT/postscripts/setupdockerhost b/xCAT/postscripts/setupdockerhost index 1babeaaa1..3ea39f33e 100755 --- a/xCAT/postscripts/setupdockerhost +++ b/xCAT/postscripts/setupdockerhost @@ -17,13 +17,38 @@ if [ "$(uname -s|tr 'A-Z' 'a-z')" = "linux" ];then fi if [[ "$OSVER" != ubuntu* ]]; then - echo "Sorry, only ubuntu have been verified at present" + echo "OS = $OSVER: Setup of Docker host is currently supported only on Ubuntu." exit 1 fi +if [[ "$ARCH" == "x86_64" ]]; then + # Check if docker-engine is installed + dpkg -l docker-engine + if [ $? -ne 0 ]; then + echo "Error: can not detect docker-engine installation." + exit 1 + fi +fi + +if [[ "$ARCH" == "ppc64el" ]]; then + # Check if docker.io is installed + dpkg -l docker.io + if [ $? -ne 0 ]; then + echo "Error: can not detect docker.io installation." + exit 1 + fi +fi + +# Docker is installed, check that it is running +docker info +if [ $? -ne 0 ]; then + echo "Docker is not running, Trying to restart." + service docker start +fi + DOCKER_VERSION=`docker info 2>/dev/null | grep "Server Version:" | awk -F: '{print $2}'` if [ -z "$DOCKER_VERSION" ];then - echo "Failed to get docker server version" + echo "Error: failed to get docker server version" exit 1 fi MAJOR_VERSION=`echo $DOCKER_VERSION | awk -F. '{print $1}'` @@ -32,7 +57,7 @@ if [ -z "$MAJOR_VERSION" -o -z "$MINOR_VERSION" ]; then echo "The docker version $DOCKER_VERSION can not be recorgnized" exit 1 elif [ $MAJOR_VERSION -le 1 -a $MINOR_VERSION -lt 10 ]; then - echo "Only docker version 1.10.x and abover is supported" + echo "Only docker version 1.10.x and above is supported" exit 1 fi @@ -42,8 +67,8 @@ mask="" gateway="" nicname="" -if [ $# -ne 1]; then - echo "Usage error, must in the format '$0 netname=net/mask@gateway[:nicname]'" +if [ $# -ne 1 ]; then + echo "Error: must be in format '$0 netname=net/mask@gateway[:nicname]'" exit 1 else NETINFO="$1" @@ -53,8 +78,15 @@ else mask=`echo $netinfo | awk -F'@' '{print $1}' | awk -F/ '{print $2}'` gateway=`echo $netinfo | awk -F'@' '{print $2}' | awk -F: '{print $1}'` nicname=`echo $netinfo | awk -F: '{print $2}'` + if [ ! -z "$nicname" ]; then + if [ ! -x /sbin/brctl ]; then + echo "Error: /sbin/brctl command not found. Not able to configure $nicname" + exit 1; + fi + fi + if [ -z "$netname" -o -z "$net" -o -z "$mask" -o -z "$gateway" ]; then - echo "Usage error, must in the format '$0 netname=net/mask@gateway[:nicname]'" + echo "Error: must be in format '$0 netname=net/mask@gateway[:nicname]'" exit 1 fi fi @@ -63,7 +95,7 @@ fi #Setup TLS master=$MASTER if ! ping $master -c 1 > /dev/null 2>&1 ; then - echo "Host $master is not reachable" + echo "Error: Host $master is not reachable" exit 1 fi @@ -87,20 +119,20 @@ if [ $? -ne 0 ]; then cat /tmp/xcat_dockerhost_cert | cat /tmp/xcat_dockerhost_cert | grep -E -v '|/{0,1}data>||' >$HOST_CERT_PEM else - echo "Can not get dockerhost certificate files" + echo "Error: can not get dockerhost certificate files" exit 1 fi cp /xcatpost/ca/ca-cert.pem $HOST_CA_PEM if [ ! -e $HOST_CA_PEM -o ! -e $HOST_CERT_PEM ];then - echo "Can not get dockerhost certificate files" + echo "Error: can not get dockerhost certificate files" exit 1 fi docker_conf_file="/etc/default/docker" if [ ! -f "$docker_conf_file" ]; then - echo "Error: file $docker_conf_file not exist" + echo "Error: file $docker_conf_file does not exist" exit 1 fi if ! grep "^DOCKER_OPTS" $docker_conf_file > /dev/null 2>&1 ; then @@ -120,12 +152,15 @@ else fi #Restart docker service -service docker restart +service docker stop +sleep 2 +systemctl daemon-reload +service docker start # It is found that sometimes the docker ps will failed if run immediately after restart docker daemon sleep 2 docker ps if [ $? -ne 0 ]; then - echo "Docker service starting failed" + echo "Error: Failed to start Docker service" exit 1 fi @@ -133,7 +168,7 @@ echo "Will create network based on: $net/$mask@$gateway===$nicname:$netname===== ret=`docker network create --gateway=$gateway --subnet=$net/$mask -o "com.docker.network.bridge.host_binding_ipv4"="$gateway" -o "com.docker.network.bridge.name"="$netname" $netname 2>&1` if [ $? -ne 0 ]; then - echo "Create network object \"$netname\" failed: $ret" + echo "Error: Creation of network object \"$netname\" failed: $ret" exit 1 fi if [ ! -z "$nicname" ]; then