From 1356b06fb11b9af0bdeba548c3cd0fc31e34a66a Mon Sep 17 00:00:00 2001 From: bybai Date: Wed, 2 Sep 2015 04:53:35 -0400 Subject: [PATCH 1/8] add manage vm doc --- .../manage_clusters/common/kvm/manage_vm.rst | 318 +++++++++++++++++- 1 file changed, 316 insertions(+), 2 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst index ab5ec9ab7..f87abd3ad 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst @@ -4,5 +4,319 @@ Manage Virtual Machine Create Virtual Machine ---------------------- -Change Virtual Machine ----------------------- +Define Virtual node "vm1" +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Define virtual machine vm1, add it to xCAT under the vm group, its ip is x.x.x.x, use makehost to add hostname and ip into /etc/hosts file: :: + + mkdef vm1 groups=vm,all + chdef vm1 ip=x.x.x.x + makehosts vm1 + +Update DNS with this new node: :: + + makedns -n + makedns -a + +Define the attributes of virtual machine +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run the chdef command to change the following attributes for the vm1: + +1. Define the virtual cpu number: :: + + chdef vm1 vmcpus=2 + +2. Define the kvm hypervisor of the virtual machine vm1, it should be set to node001: :: + + chdef vm1 vmhost=node001 + +3. Define the virtual memory size, the unit is Megabit, for example, define 1G memory to the vm1: :: + + chdef vm1 vmmemory=1024 + + Note: For diskless node, the vmmemory should be set larger than 2048, otherwise the node cannot be booted up. + +4. Define the hardware management module: :: + + chdef vm1 mgt=kvm + +5. Define the virtual network card, it should be set to the bridge br0/virb0/default which defined in hypervisor. If no bridge was set explicitly, no network device will be created for the node vm1: :: + + chdef vm1 vmnics=br0 + +6. The vmnicnicmodel attribute is used to set the type and corresponding driver for the nic. If not set, the default value is 'virtio'. + :: + + chdef vm1 vmnicnicmodel=virtio + +7. Define the storage for the vm1, three formats for the storage source are supported. + + A. Create storage on a nfs server. + The format is 'nfs:///dir', that means the kvm disk files will be created at 'nfs:///dir': :: + + chdef vm1 vmstorage=nfs:///install/vms/ + + B. Create storage on a device of hypervisor + + The format is 'phy:/dev/sdb1': :: + + chdef vm1 vmstorage=phy:/dev/sdb1 + + C. Create storage on a directory of hypervisor + + The format is 'dir:/install/vms': :: + + chdef vm1 vmstorage=dir:///install/vms + + Note: The attribute vmstorage is only necessary for diskfull node. You can ignore it for diskless node. + +8. Define the console attributes for the virtual machine: :: + + chdef vm1 serialport=0 serialspeed=115200 + +9. (optional)For monitor the installing process from kimchi, set vidpassword value: :: + + chtab node=vm1 vm.vidpassword=abc123 + +10. Set 'netboot' attribute + + * **[x86_64]** + + :: + + chdef vm1 netboot=xnba + + * **[PPC64LE]** + :: + + chdef vm1 netboot=grub2 + + Make sure the grub2 had been installed on your Management Node: :: + + rpm -aq | grep grub2 + grub2-xcat-1.0-1.noarch + + Note: If you are working with xCAT-dep oldder than 20141012, the modules for xCAT shipped grub2 can not support ubuntu LE smoothly. So the following steps needed to complete the grub2 setting. :: + + rm /tftpboot/boot/grub2/grub2.ppc + cp /tftpboot/boot/grub2/powerpc-ieee1275/core.elf /tftpboot/boot/grub2/grub2.ppc + /bin/cp -rf /tmp/iso/boot/grub/powerpc-ieee1275/elf.mod /tftpboot/boot/grub2/powerpc-ieee1275/ + +Create the virtual machine +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Create the virtual machine kvm1 with 20G hard disk. :: + + mkvm vm1 -s 20G + +If the vm1 was created successfully, a hard disk file named vm1.hda.qcow2 can be found in vmstorage location. And you can run the lsdef vm1 to see whether the mac attribute has been set automatically. + +Create osimage object +^^^^^^^^^^^^^^^^^^^^^ + +After you download the OS ISO, refer to :ref:`create_img` to create osimage objects. + +Configure password for root in xCAT MN +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +:: + + chtab key=system passwd.username=root passwd.password=xxxxxx + +Configure DHCP +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +:: + + makedhcp -n + makedhcp -a + +Set the boot state +^^^^^^^^^^^^^^^^^^^ +:: + + nodeset vm1 osimage= + +Power on the virtual machine to start OS installation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + rpower vm1 on + +If the vm1 was powered on successfully, you can get following information when running 'virsh list' on the kvm hypervisor node001. :: + + virsh list + Id Name State + -------------------------------- + 6 vm1 running + + +Use console to monitor the installing process +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +You can use console in xcat management node or kvm hypervisor to monitor the process. + +* On the kvm hypervisor you can use virsh to open text console: :: + + virsh console vm1 + +* Use rcons/wcons on the xCAT management node to open text console: :: + + makeconservercf vm1 + rcons vm1 + wcons vm1 + +* Connecting to the virtual machine's vnc console + + In order to connect to the virtual machine's console, you need to generate a new set of credentials. You can do it by running: :: + + xcatclient getrvidparms vm1 + vm1: method: kvm + vm1: textconsole: /dev/pts/0 + vm1: password: JOQTUtn0dUOBv9o3 + vm1: vidproto: vnc + vm1: server: kvmhost1 + vm1: vidport: 5900 + + Note: Now just pick your favorite vnc client and connect to the hypervisor, using the password generated by "getrvidparms". If the vnc client complains the password is not valid, it is possible that your hypervisor and headnode clocks are out of sync! You can sync them by running "ntpdate " on both the hypervisor and the headnode. + + +* Use wvid on the xCAT management node + + Make sure firewalld service had been stopped. :: + + chkconfig firewalld off + + Note: Forwarding request to systemctl will disable firewalld.service. :: + + rm /etc/systemd/system/basic.target.wants/firewalld.service + rm /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service + + Then, run wvid vm1 on MN:: + + wvid vm1 + +* For powerKVM, we can use kimchi to monitor the installing process + + Open "https://:8001" to open kimchi. There will be a “connect” button you can use below "Actions" button and input Password required:abc123 your have set before mkvm, then you could get the console. + + +Remove a virtual machine +------------------------ + +Remove the kvm1 even when it is in power on status. :: + + rmvm mv1 -f + +Remove the definition of kvm and related storage. :: + + rmvm vm1 -p + + +Clone a kvm node +---------------- + +Clone is a concept that create a new node from the old one by reuse most of data that has been installed on the old node. Before creating a new node, a vm (virtual machine) master must be created first. The new node will be created from the vm master. The new node can attach to the vm master or not. +The node can NOT be run without the vm master if choosing to make the node attach to the vm master. The advantage is that the less disk space is needed. + +**In attaching mode** + +In this mode, all the nodes will be attached to the vm master. Lesser disk space will be used than the general node. +Create the vm master kvmm from a node (vm1) and make the original node kvm2 attaches to the new created vm master: :: + + clonevm vm1 -t kvmm + vm1: Cloning vm1.hda.qcow2 (currently is 1050.6640625 MB and has a capacity of 4096MB) + vm1: Cloning of vm1.hda.qcow2 complete (clone uses 1006.74609375 for a disk size of 4096MB) + vm1: Rebasing vm1.hda.qcow2 from master + vm1: Rebased vm1.hda.qcow2 from master + +After the performing, you can see the following entry has been added into the vmmaster table. :: + + tabdump vmmaster + name,os,arch,profile,storage,storagemodel,nics,vintage,originator,comments,disable + "kvmm","rhels6","x86_64","compute","nfs:///vms/kvm",,"br0","Tue Nov 23 04:18:17 2010","root",, + +Clone a new node vm2 from vm master kvmm: :: + + clonevm vm2 -b kvmm + +**In detaching mode** + +Create a vm master that the original node detaches with the created vm master. :: + + clonevm vm2 -t kvmmd -d + vm2: Cloning vm2.hda.qcow2 (currently is 1049.4765625 MB and has a capacity of 4096MB) + vm2: Cloning of vm2.hda.qcow2 complete (clone uses 1042.21875 for a disk size of 4096MB) + +Clone the vm3 from the kvmmd with the detaching mode turn on: :: + + clonevm vm3 -b kvmmd -d + vm3: Cloning kvmmd.hda.qcow2 (currently is 1042.21875 MB and has a capacity of 4096MB) + +FAQ +--- + +1, libvirtd run into problem + + **Issue**: One error as following message: :: + + rpower kvm1 on + kvm1: internal error no supported architecture for os type 'hvm' + + **Solution**: This error was fixed by restarting libvirtd on the host machine: :: + + xdsh kvmhost1 service libvirtd restart + + Note: In any case that you find there is libvirtd error message in syslog, you can try to restart the libvirtd. + +2, Virtual disk has problem + + **Issue**: When running command 'rpower kvm1 on', get the following error message: :: + + kvm1: Error: unable to set user and group to '0:0' + on '/var/lib/xcat/pools/27f1df4b-e6cb-5ed2-42f2-9ef7bdd5f00f/kvm1.hda.qcow2': Invalid argument: + + **Solution**: try to figure out the nfs:// server was exported correctly. The nfs client should have root authority. + +3, VNC client complains the credentials are not valid + + **Issue**: When connecting to the hypervisor using VNC to get a VM console, the vnc client complains with "Authentication failed". + + **Solution**: Check if the clocks on your hypervisor and headnode are in sync! + +4, rpower fails with "qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vmXYZ.hda.qcow2: Permission denied" error message + + **Issue**: When running rpower on a kvm vm, rpower complains with the following error message: :: + + rpower vm1 on + vm1: Error: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 + qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vm1.hda.qcow2: Permission denied: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 + qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vm1.hda.qcow2: Permission denied + [root@xcat xCAT_plugin]# + + **Solution**: This might be caused by bad permissions in your NFS server / client (where clients will not mount the share with the correct permissions). Systems like CentOS 6 will have NFS v4 support activated by default. This might be causing the above mentioned problems so one solution is to simply disable NFS v4 support in your NFS server by uncommenting the following option in /etc/sysconfig/nfs: :: + + RPCNFSDARGS="-N 4" + + Finish by restarting your NFS services (i.e. service nfsd restart) and try powering on your VM again... + Note: if you are running a stateless hypervisor, we advise you to purge the VM (rmvm -p vmXYZ), restart the hypervisor and "mkvm vmXYZ -s 4" to recreate the VM as soon as the hypervisor is up and running. + +5, Error: Cannot communicate via libvirt to + + **Issue**: This error mostly caused by the incorrect setting of the ssh tunnel between xCAT management node and . + + **Solution**: Check that xCAT MN could ssh to the without password. + +6, Cannot ping to the vm after the first boot of stateful install + + **Issue**: The new installed stateful vm node is not pingable after the first boot, you may see the following error message in the console when vm booting: :: + + ADDRCONF(NETDEV_UP): eth0 link is not ready. + + **Solutoin**: This issue may be caused by the incorrect driver for vm. You can try to change driver to 'virtio' by following steps: :: + + rmvm kvm1 + chdef kvm1 vmnicnicmodel=virtio + mkvm kvm1 + + From 1ef31f1418f3c4a2e7c02cfd6d987a17b6d9a179 Mon Sep 17 00:00:00 2001 From: bybai Date: Wed, 2 Sep 2015 04:54:53 -0400 Subject: [PATCH 2/8] add _diskful_installation label --- .../admin-guides/manage_clusters/ppc64le/diskful/index.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskful/index.rst b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskful/index.rst index 3e7c62d47..4b3907c02 100644 --- a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskful/index.rst +++ b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskful/index.rst @@ -1,3 +1,5 @@ +.. _diskful_installation: + Diskful Installation ==================== From 63101586e2697b426f94ab0115f00061f32cf2a7 Mon Sep 17 00:00:00 2001 From: bybai Date: Wed, 2 Sep 2015 04:55:21 -0400 Subject: [PATCH 3/8] add powerKVM doc --- .../ppc64le/virtual_machines/powerKVM.rst | 27 +++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/ppc64le/virtual_machines/powerKVM.rst b/docs/source/guides/admin-guides/manage_clusters/ppc64le/virtual_machines/powerKVM.rst index 4c334720f..613070857 100644 --- a/docs/source/guides/admin-guides/manage_clusters/ppc64le/virtual_machines/powerKVM.rst +++ b/docs/source/guides/admin-guides/manage_clusters/ppc64le/virtual_machines/powerKVM.rst @@ -1,5 +1,28 @@ PowerKVM ======== -Install PoweKVM ---------------- +Install PowerKVM +---------------- + +The process to set up PowerKVM hypervisor with xCAT is the same with Diskfull installation. Prepare powerKVM iso, such as ibm-powerkvm-2.1.1.0-22.0-ppc64-gold-201410191558.iso, then refer to :ref:`diskful_installation` to install PowerKVM hypervisor. + +Check bridge setting after installation finished +------------------------------------------------ + +After PowerKVM hypervisor is installed successfully, you can get the bridge information: :: + + # brctl show + bridge name bridge id STP enabled interfaces + br0 8000.000000000000 no eth0 + +If the bridge show is not like above, it means that you may not run xCAT post install script. You can manually run following commands to create the bridge, for example: :: + + IPADDR=10.1.101.1/16 + brctl addbr br0 + brctl addif br0 eth0 + brctl setfd br0 0 + ip addr add dev br0 $IPADDR + ip link set br0 up + ip addr del dev eth0 $IPADDR + +Note: During ubuntu LE virtual machines installation, the virtual machines need to access Internet, so make sure the PowerKVM hypervisor is able to access Internet. From 555f440d7a52aa6d5ee9172439a62b94df87a0ce Mon Sep 17 00:00:00 2001 From: bybai Date: Sun, 6 Sep 2015 15:59:07 +0800 Subject: [PATCH 4/8] polished updates for mkvm and console --- .../admin-guides/manage_clusters/common/kvm/manage_vm.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst index f87abd3ad..1592353f3 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst @@ -75,7 +75,7 @@ Run the chdef command to change the following attributes for the vm1: chdef vm1 serialport=0 serialspeed=115200 -9. (optional)For monitor the installing process from kimchi, set vidpassword value: :: +9. (optional)For monitor the installing process from vnc client, set vidpassword value: :: chtab node=vm1 vm.vidpassword=abc123 @@ -105,8 +105,11 @@ Run the chdef command to change the following attributes for the vm1: Create the virtual machine ^^^^^^^^^^^^^^^^^^^^^^^^^^ +If vmstorage is on a nfs server or a device of hypervisor, for example :: -Create the virtual machine kvm1 with 20G hard disk. :: + mkvm vm1 + +If create the virtual machine kvm1 with 20G hard disk from a large disk directory, for example :: mkvm vm1 -s 20G @@ -162,6 +165,7 @@ You can use console in xcat management node or kvm hypervisor to monitor the pro * Use rcons/wcons on the xCAT management node to open text console: :: + cons=kvm makeconservercf vm1 rcons vm1 wcons vm1 From 6e82f209e0373b19a7c83889cc953152e5f77e9f Mon Sep 17 00:00:00 2001 From: bybai Date: Wed, 9 Sep 2015 23:12:59 -0400 Subject: [PATCH 5/8] polished according to comments. --- .../ppc64le/virtual_machines/powerKVM.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/ppc64le/virtual_machines/powerKVM.rst b/docs/source/guides/admin-guides/manage_clusters/ppc64le/virtual_machines/powerKVM.rst index 613070857..cef4e2e05 100644 --- a/docs/source/guides/admin-guides/manage_clusters/ppc64le/virtual_machines/powerKVM.rst +++ b/docs/source/guides/admin-guides/manage_clusters/ppc64le/virtual_machines/powerKVM.rst @@ -6,16 +6,16 @@ Install PowerKVM The process to set up PowerKVM hypervisor with xCAT is the same with Diskfull installation. Prepare powerKVM iso, such as ibm-powerkvm-2.1.1.0-22.0-ppc64-gold-201410191558.iso, then refer to :ref:`diskful_installation` to install PowerKVM hypervisor. -Check bridge setting after installation finished ------------------------------------------------- +Verifying hypervisor bridges +---------------------------- -After PowerKVM hypervisor is installed successfully, you can get the bridge information: :: +After PowerKVM hypervisor is installed successfully, you can get the bridge information by running ``brctl show``: :: # brctl show bridge name bridge id STP enabled interfaces br0 8000.000000000000 no eth0 -If the bridge show is not like above, it means that you may not run xCAT post install script. You can manually run following commands to create the bridge, for example: :: +If there are no bridges configured, the xCAT post install script will not work. You must manually create a bridge. The following is provided as an example for creating a bridge bro using interface eth0 with IP address: 10.1.101.1/16, for example: :: IPADDR=10.1.101.1/16 brctl addbr br0 @@ -25,4 +25,4 @@ If the bridge show is not like above, it means that you may not run xCAT post in ip link set br0 up ip addr del dev eth0 $IPADDR -Note: During ubuntu LE virtual machines installation, the virtual machines need to access Internet, so make sure the PowerKVM hypervisor is able to access Internet. +Note: During any of ubuntu installation, the virtual machines need to access Internet, so make sure the PowerKVM hypervisor is able to access Internet. From 6d2ce0ab53ccbb116678f4be0a406bb3b8e34117 Mon Sep 17 00:00:00 2001 From: bybai Date: Thu, 10 Sep 2015 00:39:32 -0400 Subject: [PATCH 6/8] polished the docs --- .../manage_clusters/common/kvm/manage_vm.rst | 55 +++++++++---------- 1 file changed, 26 insertions(+), 29 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst index 1592353f3..3b6e7f34d 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst @@ -1,10 +1,12 @@ -Manage Virtual Machine -====================== +Manage Virtual Machine (VMs) +============================ -Create Virtual Machine +Create the Virtual Machine ---------------------- -Define Virtual node "vm1" +In this doc, we assume the powerKVM hypervisor host node001 is ready to use. + +Create VM Node Definition ^^^^^^^^^^^^^^^^^^^^^^^^^ Define virtual machine vm1, add it to xCAT under the vm group, its ip is x.x.x.x, use makehost to add hostname and ip into /etc/hosts file: :: @@ -18,8 +20,8 @@ Update DNS with this new node: :: makedns -n makedns -a -Define the attributes of virtual machine -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Define attributes for the VM +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Run the chdef command to change the following attributes for the vm1: @@ -31,7 +33,7 @@ Run the chdef command to change the following attributes for the vm1: chdef vm1 vmhost=node001 -3. Define the virtual memory size, the unit is Megabit, for example, define 1G memory to the vm1: :: +3. Define the virtual memory size, the unit is Megabit. For example, to define 1GB of memory to vm1: :: chdef vm1 vmmemory=1024 @@ -52,20 +54,21 @@ Run the chdef command to change the following attributes for the vm1: 7. Define the storage for the vm1, three formats for the storage source are supported. - A. Create storage on a nfs server. - The format is 'nfs:///dir', that means the kvm disk files will be created at 'nfs:///dir': :: + A. Create storage on a nfs server - chdef vm1 vmstorage=nfs:///install/vms/ + The format is ``nfs:///dir``, that means the kvm disk files will be created at ``nfs:///dir``: :: + + chdef vm1 vmstorage=nfs:///install/vms/ B. Create storage on a device of hypervisor - The format is 'phy:/dev/sdb1': :: + Instead of the format is 'phy:/dev/sdb1': :: chdef vm1 vmstorage=phy:/dev/sdb1 C. Create storage on a directory of hypervisor - The format is 'dir:/install/vms': :: + Instead of he format is 'dir:/install/vms': :: chdef vm1 vmstorage=dir:///install/vms @@ -103,8 +106,8 @@ Run the chdef command to change the following attributes for the vm1: cp /tftpboot/boot/grub2/powerpc-ieee1275/core.elf /tftpboot/boot/grub2/grub2.ppc /bin/cp -rf /tmp/iso/boot/grub/powerpc-ieee1275/elf.mod /tftpboot/boot/grub2/powerpc-ieee1275/ -Create the virtual machine -^^^^^^^^^^^^^^^^^^^^^^^^^^ +Make the VM under xCAT +^^^^^^^^^^^^^^^^^^^^^^ If vmstorage is on a nfs server or a device of hypervisor, for example :: mkvm vm1 @@ -120,12 +123,6 @@ Create osimage object After you download the OS ISO, refer to :ref:`create_img` to create osimage objects. -Configure password for root in xCAT MN -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -:: - - chtab key=system passwd.username=root passwd.password=xxxxxx - Configure DHCP ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :: @@ -133,14 +130,14 @@ Configure DHCP makedhcp -n makedhcp -a -Set the boot state -^^^^^^^^^^^^^^^^^^^ +Prepare the VM for installation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :: nodeset vm1 osimage= -Power on the virtual machine to start OS installation -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Start VM Installation +^^^^^^^^^^^^^^^^^^^^^ :: @@ -154,8 +151,8 @@ If the vm1 was powered on successfully, you can get following information when r 6 vm1 running -Use console to monitor the installing process -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Monitoring the Virtual Machine +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You can use console in xcat management node or kvm hypervisor to monitor the process. @@ -205,7 +202,7 @@ You can use console in xcat management node or kvm hypervisor to monitor the pro Open "https://:8001" to open kimchi. There will be a “connect” button you can use below "Actions" button and input Password required:abc123 your have set before mkvm, then you could get the console. -Remove a virtual machine +Remove the virtual machine ------------------------ Remove the kvm1 even when it is in power on status. :: @@ -217,8 +214,8 @@ Remove the definition of kvm and related storage. :: rmvm vm1 -p -Clone a kvm node ----------------- +Clone the virtual machine +------------------------- Clone is a concept that create a new node from the old one by reuse most of data that has been installed on the old node. Before creating a new node, a vm (virtual machine) master must be created first. The new node will be created from the vm master. The new node can attach to the vm master or not. The node can NOT be run without the vm master if choosing to make the node attach to the vm master. The advantage is that the less disk space is needed. From e85523c35e92f6acc246c7bee9afdfaaabd57bde Mon Sep 17 00:00:00 2001 From: bybai Date: Fri, 11 Sep 2015 04:33:19 -0400 Subject: [PATCH 7/8] polished based on comments. --- .../manage_clusters/common/kvm/manage_vm.rst | 60 ++++++++++--------- 1 file changed, 31 insertions(+), 29 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst index 3b6e7f34d..538f60b6b 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst @@ -7,7 +7,7 @@ Create the Virtual Machine In this doc, we assume the powerKVM hypervisor host node001 is ready to use. Create VM Node Definition -^^^^^^^^^^^^^^^^^^^^^^^^^ +````````````````````````` Define virtual machine vm1, add it to xCAT under the vm group, its ip is x.x.x.x, use makehost to add hostname and ip into /etc/hosts file: :: @@ -21,7 +21,7 @@ Update DNS with this new node: :: makedns -a Define attributes for the VM -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +````````````````````````````` Run the chdef command to change the following attributes for the vm1: @@ -107,37 +107,40 @@ Run the chdef command to change the following attributes for the vm1: /bin/cp -rf /tmp/iso/boot/grub/powerpc-ieee1275/elf.mod /tftpboot/boot/grub2/powerpc-ieee1275/ Make the VM under xCAT -^^^^^^^^^^^^^^^^^^^^^^ +`````````````````````` + If vmstorage is on a nfs server or a device of hypervisor, for example :: mkvm vm1 -If create the virtual machine kvm1 with 20G hard disk from a large disk directory, for example :: +If create the virtual machine vm1 with 20G hard disk from a large disk directory, for example :: mkvm vm1 -s 20G -If the vm1 was created successfully, a hard disk file named vm1.hda.qcow2 can be found in vmstorage location. And you can run the lsdef vm1 to see whether the mac attribute has been set automatically. +If the vm1 was created successfully, a hard disk file named vm1.sda.qcow2 can be found in vmstorage location. And you can run the lsdef vm1 to see whether the mac attribute has been set automatically. -Create osimage object -^^^^^^^^^^^^^^^^^^^^^ - -After you download the OS ISO, refer to :ref:`create_img` to create osimage objects. Configure DHCP -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +``````````````` :: makedhcp -n makedhcp -a +Create osimage object +`````````````````````````````` + +After you download the OS ISO, refer to :ref:`create_img` to create osimage objects. + + Prepare the VM for installation -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +``````````````````````````````````````` :: nodeset vm1 osimage= Start VM Installation -^^^^^^^^^^^^^^^^^^^^^ +`````````````````````` :: @@ -152,7 +155,7 @@ If the vm1 was powered on successfully, you can get following information when r Monitoring the Virtual Machine -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +`````````````````````````````` You can use console in xcat management node or kvm hypervisor to monitor the process. @@ -162,10 +165,9 @@ You can use console in xcat management node or kvm hypervisor to monitor the pro * Use rcons/wcons on the xCAT management node to open text console: :: - cons=kvm + chdef vm1 cons=kvm makeconservercf vm1 rcons vm1 - wcons vm1 * Connecting to the virtual machine's vnc console @@ -176,7 +178,7 @@ You can use console in xcat management node or kvm hypervisor to monitor the pro vm1: textconsole: /dev/pts/0 vm1: password: JOQTUtn0dUOBv9o3 vm1: vidproto: vnc - vm1: server: kvmhost1 + vm1: server: node001 vm1: vidport: 5900 Note: Now just pick your favorite vnc client and connect to the hypervisor, using the password generated by "getrvidparms". If the vnc client complains the password is not valid, it is possible that your hypervisor and headnode clocks are out of sync! You can sync them by running "ntpdate " on both the hypervisor and the headnode. @@ -226,16 +228,16 @@ In this mode, all the nodes will be attached to the vm master. Lesser disk space Create the vm master kvmm from a node (vm1) and make the original node kvm2 attaches to the new created vm master: :: clonevm vm1 -t kvmm - vm1: Cloning vm1.hda.qcow2 (currently is 1050.6640625 MB and has a capacity of 4096MB) - vm1: Cloning of vm1.hda.qcow2 complete (clone uses 1006.74609375 for a disk size of 4096MB) - vm1: Rebasing vm1.hda.qcow2 from master - vm1: Rebased vm1.hda.qcow2 from master + vm1: Cloning vm1.sda.qcow2 (currently is 1050.6640625 MB and has a capacity of 4096MB) + vm1: Cloning of vm1.sda.qcow2 complete (clone uses 1006.74609375 for a disk size of 4096MB) + vm1: Rebasing vm1.sda.qcow2 from master + vm1: Rebased vm1.sda.qcow2 from master After the performing, you can see the following entry has been added into the vmmaster table. :: tabdump vmmaster name,os,arch,profile,storage,storagemodel,nics,vintage,originator,comments,disable - "kvmm","rhels6","x86_64","compute","nfs:///vms/kvm",,"br0","Tue Nov 23 04:18:17 2010","root",, + "kvmm","","","compute","nfs:///vms/kvm",,"br0","","root",, Clone a new node vm2 from vm master kvmm: :: @@ -246,13 +248,13 @@ Clone a new node vm2 from vm master kvmm: :: Create a vm master that the original node detaches with the created vm master. :: clonevm vm2 -t kvmmd -d - vm2: Cloning vm2.hda.qcow2 (currently is 1049.4765625 MB and has a capacity of 4096MB) - vm2: Cloning of vm2.hda.qcow2 complete (clone uses 1042.21875 for a disk size of 4096MB) + vm2: Cloning vm2.sda.qcow2 (currently is 1049.4765625 MB and has a capacity of 4096MB) + vm2: Cloning of vm2.sda.qcow2 complete (clone uses 1042.21875 for a disk size of 4096MB) Clone the vm3 from the kvmmd with the detaching mode turn on: :: clonevm vm3 -b kvmmd -d - vm3: Cloning kvmmd.hda.qcow2 (currently is 1042.21875 MB and has a capacity of 4096MB) + vm3: Cloning kvmmd.sda.qcow2 (currently is 1042.21875 MB and has a capacity of 4096MB) FAQ --- @@ -275,9 +277,9 @@ FAQ **Issue**: When running command 'rpower kvm1 on', get the following error message: :: kvm1: Error: unable to set user and group to '0:0' - on '/var/lib/xcat/pools/27f1df4b-e6cb-5ed2-42f2-9ef7bdd5f00f/kvm1.hda.qcow2': Invalid argument: + on '/var/lib/xcat/pools/27f1df4b-e6cb-5ed2-42f2-9ef7bdd5f00f/kvm1.sda.qcow2': Invalid argument: - **Solution**: try to figure out the nfs:// server was exported correctly. The nfs client should have root authority. + **Solution**: try to figure out the ``nfs://`` was exported correctly. The nfs client should have root authority. 3, VNC client complains the credentials are not valid @@ -285,14 +287,14 @@ FAQ **Solution**: Check if the clocks on your hypervisor and headnode are in sync! -4, rpower fails with "qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vmXYZ.hda.qcow2: Permission denied" error message +4, rpower fails with "qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vmXYZ.sda.qcow2: Permission denied" error message **Issue**: When running rpower on a kvm vm, rpower complains with the following error message: :: rpower vm1 on vm1: Error: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 - qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vm1.hda.qcow2: Permission denied: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 - qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vm1.hda.qcow2: Permission denied + qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vm1.sda.qcow2: Permission denied: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 + qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vm1.sda.qcow2: Permission denied [root@xcat xCAT_plugin]# **Solution**: This might be caused by bad permissions in your NFS server / client (where clients will not mount the share with the correct permissions). Systems like CentOS 6 will have NFS v4 support activated by default. This might be causing the above mentioned problems so one solution is to simply disable NFS v4 support in your NFS server by uncommenting the following option in /etc/sysconfig/nfs: :: From 0decb6aea9b134c07e99cb9a8f1e4f208b060928 Mon Sep 17 00:00:00 2001 From: bybai Date: Mon, 14 Sep 2015 05:27:08 -0400 Subject: [PATCH 8/8] polished --- .../manage_clusters/common/kvm/manage_vm.rst | 56 +++++++++---------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst index 538f60b6b..fc6ecb547 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/kvm/manage_vm.rst @@ -4,7 +4,7 @@ Manage Virtual Machine (VMs) Create the Virtual Machine ---------------------- -In this doc, we assume the powerKVM hypervisor host node001 is ready to use. +In this doc, we assume the powerKVM hypervisor host cn1 is ready to use. Create VM Node Definition ````````````````````````` @@ -29,9 +29,9 @@ Run the chdef command to change the following attributes for the vm1: chdef vm1 vmcpus=2 -2. Define the kvm hypervisor of the virtual machine vm1, it should be set to node001: :: +2. Define the kvm hypervisor of the virtual machine vm1, it should be set to cn1: :: - chdef vm1 vmhost=node001 + chdef vm1 vmhost=cn1 3. Define the virtual memory size, the unit is Megabit. For example, to define 1GB of memory to vm1: :: @@ -68,9 +68,9 @@ Run the chdef command to change the following attributes for the vm1: C. Create storage on a directory of hypervisor - Instead of he format is 'dir:/install/vms': :: + Instead of he format is 'dir:///var/lib/libvirt/images': :: - chdef vm1 vmstorage=dir:///install/vms + chdef vm1 vmstorage=dir:///var/lib/libvirt/images Note: The attribute vmstorage is only necessary for diskfull node. You can ignore it for diskless node. @@ -146,7 +146,7 @@ Start VM Installation rpower vm1 on -If the vm1 was powered on successfully, you can get following information when running 'virsh list' on the kvm hypervisor node001. :: +If the vm1 was powered on successfully, you can get following information when running 'virsh list' on the kvm hypervisor cn1. :: virsh list Id Name State @@ -178,7 +178,7 @@ You can use console in xcat management node or kvm hypervisor to monitor the pro vm1: textconsole: /dev/pts/0 vm1: password: JOQTUtn0dUOBv9o3 vm1: vidproto: vnc - vm1: server: node001 + vm1: server: cn1 vm1: vidport: 5900 Note: Now just pick your favorite vnc client and connect to the hypervisor, using the password generated by "getrvidparms". If the vnc client complains the password is not valid, it is possible that your hypervisor and headnode clocks are out of sync! You can sync them by running "ntpdate " on both the hypervisor and the headnode. @@ -207,9 +207,9 @@ You can use console in xcat management node or kvm hypervisor to monitor the pro Remove the virtual machine ------------------------ -Remove the kvm1 even when it is in power on status. :: +Remove the vm1 even when it is in power on status. :: - rmvm mv1 -f + rmvm vm1 -f Remove the definition of kvm and related storage. :: @@ -225,9 +225,9 @@ The node can NOT be run without the vm master if choosing to make the node attac **In attaching mode** In this mode, all the nodes will be attached to the vm master. Lesser disk space will be used than the general node. -Create the vm master kvmm from a node (vm1) and make the original node kvm2 attaches to the new created vm master: :: +Create the vm master vm5 from a node (vm1) and make the original node vm1 attaches to the new created vm master: :: - clonevm vm1 -t kvmm + clonevm vm1 -t vm5 vm1: Cloning vm1.sda.qcow2 (currently is 1050.6640625 MB and has a capacity of 4096MB) vm1: Cloning of vm1.sda.qcow2 complete (clone uses 1006.74609375 for a disk size of 4096MB) vm1: Rebasing vm1.sda.qcow2 from master @@ -237,24 +237,24 @@ After the performing, you can see the following entry has been added into the vm tabdump vmmaster name,os,arch,profile,storage,storagemodel,nics,vintage,originator,comments,disable - "kvmm","","","compute","nfs:///vms/kvm",,"br0","","root",, + "vm5","","","compute","nfs:///vms/kvm",,"br0","","root",, -Clone a new node vm2 from vm master kvmm: :: +Clone a new node vm2 from vm master vm5: :: - clonevm vm2 -b kvmm + clonevm vm2 -b vm5 **In detaching mode** Create a vm master that the original node detaches with the created vm master. :: - clonevm vm2 -t kvmmd -d + clonevm vm2 -t vm6 -d vm2: Cloning vm2.sda.qcow2 (currently is 1049.4765625 MB and has a capacity of 4096MB) vm2: Cloning of vm2.sda.qcow2 complete (clone uses 1042.21875 for a disk size of 4096MB) -Clone the vm3 from the kvmmd with the detaching mode turn on: :: +Clone the vm3 from the vm6 with the detaching mode turn on: :: - clonevm vm3 -b kvmmd -d - vm3: Cloning kvmmd.sda.qcow2 (currently is 1042.21875 MB and has a capacity of 4096MB) + clonevm vm3 -b vm6 -d + vm3: Cloning vm6.sda.qcow2 (currently is 1042.21875 MB and has a capacity of 4096MB) FAQ --- @@ -263,21 +263,21 @@ FAQ **Issue**: One error as following message: :: - rpower kvm1 on - kvm1: internal error no supported architecture for os type 'hvm' + rpower vm1 on + vm1: internal error no supported architecture for os type 'hvm' **Solution**: This error was fixed by restarting libvirtd on the host machine: :: - xdsh kvmhost1 service libvirtd restart + xdsh cn1 service libvirtd restart Note: In any case that you find there is libvirtd error message in syslog, you can try to restart the libvirtd. 2, Virtual disk has problem - **Issue**: When running command 'rpower kvm1 on', get the following error message: :: + **Issue**: When running command 'rpower vm1 on', get the following error message: :: - kvm1: Error: unable to set user and group to '0:0' - on '/var/lib/xcat/pools/27f1df4b-e6cb-5ed2-42f2-9ef7bdd5f00f/kvm1.sda.qcow2': Invalid argument: + vm1: Error: unable to set user and group to '0:0' + on '/var/lib/xcat/pools/27f1df4b-e6cb-5ed2-42f2-9ef7bdd5f00f/vm1.sda.qcow2': Invalid argument: **Solution**: try to figure out the ``nfs://`` was exported correctly. The nfs client should have root authority. @@ -289,7 +289,7 @@ FAQ 4, rpower fails with "qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vmXYZ.sda.qcow2: Permission denied" error message - **Issue**: When running rpower on a kvm vm, rpower complains with the following error message: :: + **Issue**: When running rpower on a vm, rpower complains with the following error message: :: rpower vm1 on vm1: Error: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 @@ -318,8 +318,8 @@ FAQ **Solutoin**: This issue may be caused by the incorrect driver for vm. You can try to change driver to 'virtio' by following steps: :: - rmvm kvm1 - chdef kvm1 vmnicnicmodel=virtio - mkvm kvm1 + rmvm vm1 + chdef vm1 vmnicnicmodel=virtio + mkvm vm1