From 7c0a548775fa4eeb2f954bc0a09ec869c4b732af Mon Sep 17 00:00:00 2001 From: penguhyang Date: Wed, 25 Nov 2015 00:45:31 -0500 Subject: [PATCH] add document about vlan configuration --- docs/source/advanced/networks/index.rst | 1 - docs/source/advanced/networks/vlan/vlan.rst | 226 ++++++++++++++++++++ 2 files changed, 226 insertions(+), 1 deletion(-) diff --git a/docs/source/advanced/networks/index.rst b/docs/source/advanced/networks/index.rst index 4ac978fa4..3390f6c7d 100644 --- a/docs/source/advanced/networks/index.rst +++ b/docs/source/advanced/networks/index.rst @@ -6,4 +6,3 @@ Networks infiniband/index.rst ipv6/index.rst - vlan/index.rst diff --git a/docs/source/advanced/networks/vlan/vlan.rst b/docs/source/advanced/networks/vlan/vlan.rst index 0867106da..f1d116bb6 100644 --- a/docs/source/advanced/networks/vlan/vlan.rst +++ b/docs/source/advanced/networks/vlan/vlan.rst @@ -1,2 +1,228 @@ VLAN Configuration ================== + +Overview +-------- + +The main intent for this feature is the following scenario: you have a cluster with 100 nodes all on the Ethernet switch. A user requests 10 nodes from the cluster, but wants to run some sensitive things on the 10 nodes, so wants them isolated from the other 90 nodes. So as part of scheduling these 10 nodes for this user, you want to create a VLAN for these 10 nodes that is separate from the LAN the other 90 nodes are on. + +xCAT has provided the following commands to do the VLAN creation and manipulation in the **xCAT-vlan** package. + +* mkvlan +* chvlan +* lsvlan +* rmvlan + +These VLAN functions are supported for stand-alone nodes as well as virtual machines such as KVMs. + +Install the package +------------------- + +Install the ``xCAT-vlan`` package using the package manager on the OS. + +[RHEL] :: + + yum install xCAT-vlan + +[SLES] :: + + zypper install xCAT-vlan + +[Ubuntu] :: + + apt-get install xCAT-vlan + +Prepare the Cluster +------------------- + +Assume the management node is installed and the other nodes are defined in the cluster. The following is what you need to do in order to use the VLAN Configuration feature. + +**1. Populate the switches table** + +Make sure the correct SNMP information is entered into the **switches** table for the switches involved. If there are more than one switches involved in a vlan, the ports that connect to the switches need to entered in switches.linkports with the following format: :: + + :switch,:switch.... + +For example: :: + + # switch,snmpversion,username,password,privacy,auth,linkports,comments,disable + "switch1","3","user1","passw0rd",,"sha","42:switch2",, + "switch2","2c",,,,,"50:switch1",, + "switch3","3","admin","passw0rd","des","sha",,, + "switch4","2c",,"mycomminity",,,,, + +This means port 42 of switch1 is connected to port 50 of switch2. And switch1 can be accessed using SNMP version 3 and switch 2 can be accessed using SNMP version 2. + +Note: The **username** and the **password** on the switches table are NOT the same as SSH user name and password. You have to configure SNMP on the switch for these parameters and then fill up this table. Please use **tabdump switches -d** command to find out the meaning of each column. + +**2. Populate the switch table** + +Make sure each node has the correct switch name and port number in the **switch** table. + +For example: :: + + # node,switch,port,vlan,interface,comments,disable + "node1","switch1","33",,,, + "node2","switch1","34",,,, + "node3","switch2","10",,,, + +For xCAT-vlan 2.7.5 and later versions, it supports creating vlans for other networks as well. To specify other networks in the switch table, the switch.interface must be specified. For example: :: + + # node,switch,port,vlan,interface,comments,disable + "node1","switch1","33",,,, + "node2","switch1","34",,"primary",, + "node3","switch2","10",,"primary:eth0",, + "node1","switch2","11",,"eth1",, + "node2","switch2","12",,"eth1",, + "node3","switch2","13",,"eth1",, + +The interface eth1 is for the application network on node1, node2 and node3. Note that there are two rows for each node. One is for the management network and the other is for the application network. The value for **switch.interface** for management network can be empty, the word "primary" or "primary:ethx". + +**3. Configure the switch for SNMP access** + +Please make sure that the MN can access the switch using SNMP and the switch is configured such that it has SNMP read and write permissions. + +You can use **snmpwalk/snmpget** and **snmpset** commands on the mn to check. These commands are from **net-snmp-utils** rpm. + +**4. Define the VLAN subnet and mask pattern** + +The **site** table keys called "vlannets" and "vlanmasks" will be used to define a range of networks that can be used to define vlans. The format is a regular expression. + +For example: :: + + vlannets: |(\d+)|10.($1+0).0.0| + vlanmasks: 255.255.255.0 + +This means that the network for the vlan id 5 will be 10.5.0.0 and the mask is 255.255.255.0. + +However, user can also customize a vlan network and netmask using -t and -m flags on **mkvlan** command. + +**5. Customize host names and ip addresses for nodes** + +Within the vlan, by default the hostnames for the nodes are having the following format: :: + + vnY + +where Y is the node number. + +For example, the hostname for node 5 on vlan 10 is v10n5. + +User can customize the host name and ip addresses using the **hosts** table. If the host name and ip addresses are found on the **hosts.otherinterfaces**, then it will be used. For example: :: + + #node,ip,hostnames,otherinterfaces,comments,disable + "node1","192.168.1.1",,"test1:10.0.0.1",, + "node2","192.168.1.2",,"test2:10.0.0.2",, + +**6. For KVM clients** + +If you are going to include KVM clients in the VLANs, set the site table key "usexhrm" to be 1. :: + + chdef -t site usexhrm=1 + +Create a VLAN +------------- + +For standalone nodes, VLAN can be created while the nodes are running or down. + +To make a private vlan for stand-alone nodes for the management network: :: + + mkvlan -n node1,node2,node3 + +You can specify vlan id, subnet and netmask etc. :: + + mkvlan 3 -n node1,node2,node3 -t 10.3.2.0 -m 255.255.255.0 + +For virtual machines, the vm guests must be down. To make a private vlan for KVM guests. :: + + chdef -t site -o clustersite usexhrm=1 + mkdef node1 arch=x86_64 groups=kvm,all installnic=mac primarynic=mac mgt=kvm netboot=pxe nfsserver=10.1.0.204 os=rhels6 profile=compute provmethod=install serialport=0 serialspeed=115200 vmcpus=1 vmhost=x3650n01 vmmemory=512 vmnics=br0 vmstorage=nfs://10.1.0.203/vms + mkdef node2 arch=x86_64 groups=kvm,all installnic=mac primarynic=mac mgt=kvm netboot=pxe nfsserver=10.1.0.204 os=rhels6 profile=compute provmethod=install serialport=0 serialspeed=115200 vmcpus=1 vmhost=x3650n01 vmmemory=512 vmnics=br0 vmstorage=nfs://10.1.0.203/vms + mkvlan -n node1,node2 + mkvm node1,node2 -s 20G + rpower node1,node2 on + rinstall node1,node2 + +For xCAT-vlan 2.7.5 and later versions, you can create vlans for other networks. This can be done by using -i flag to specify the interface of the network. For example: :: + + mkvlan -n node1,node2,node3 -i eth1 + +A tagged vlan will be created for the network that is on eth1 for node1, node2 and node3. For KVM clients, -i specifies the interface name on the KVM host that the vlan will be tagged on. If -i is omitted, the management networks will be assumed. + +Note: After the vlan is created, the nodes can still be accessed by the mn using the management network. You can use **lsvan** command to list all the vlans. + +For example: :: + + # lsvlan + vlan 3: + subnet 10.3.0.0 + netmask 255.255.0.0 + vlan 99: + subnet 10.99.0.0 + netmask 255.255.0.0 + + # lsvlan 3 + vlan 3 + subnet 10.3.0.0 + netmask 255.255.0.0 + hostname ip address node vm host + v3n1 10.3.0.1 node1 + v3n2 10.3.0.2 node2 + v3n3 10.3.0.3 node3 host1 + +Modify a VLAN +------------- + +You can use the **chvlan** command to add or remove nodes to/from an existing vlan. + +For standalone nodes, just run the command while the node are running or not. For example: + +To add :: + + chvlan 3 -n node4,node5 + +To remove :: + + chvlan 3 -n node4,node5 -d + +For virtual machines, adding them to the vlan requires that they are defined and they are not up and running. + +For example: :: + + mkdef node4 arch=x86_64 groups=kvm,all installnic=mac primarynic=mac mgt=kvm netboot=pxe nfsserver=10.1.0.204 os=rhels6 profile=compute provmethod=install serialport=0 serialspeed=115200 vmcpus=1 vmhost=x3650n01 vmmemory=512 vmnics=br0 vmstorage=nfs://10.1.0.203/vms + mkdef node5 arch=x86_64 groups=kvm,all installnic=mac primarynic=mac mgt=kvm netboot=pxe nfsserver=10.1.0.204 os=rhels6 profile=compute provmethod=install serialport=0 serialspeed=115200 vmcpus=1 vmhost=x3650n01 vmmemory=512 vmnics=br0 vmstorage=nfs://10.1.0.203/vms + chvlan 3 -n node4,node5 + mkvm node4,node5 -s 20G + rpower node4,node5 on + rinstall node4,node5 + +For xCAT-vlan 2.7.5 and later versions, you can modify vlans for other networks. This can be done by using -i flag to specify the interface of the network. For KVM clients, -i specifies the interface name on the KVM host that the vlan will be tagged on. If -i is omitted, the management networks will be assumed. + +For example: :: + + chvlan 3 -n node4,node5 -i eth1 + +There is no need to specify -i flag for removing nodes from a vlan. + +Remove a VLAN +------------- + +The **rmvlan** command removes the given vlan ID from the cluster. It removes the vlan id from all the swithces involved, deconfigures the nodes so that vlan adaptor (tag) will be remved, cleans up /etc/hosts, DNS and database tables for the given vlan. + +For example: :: + + rmvlan 3 + +VLAN Security +------------- + +To make the vlan more secure, the root guard and the bpdu guard are enabled for each ports within the vlan by **mkvlan** and **chvlan** commands. This way it guards the topology changes on the switch by the hackers who hack the STP. However, when the vlan is removed by the **rmvlan** and the **chvlan (-d)** commands, the root guard and the bpdu guard are not disabled because the code cannot tell if the guards were enabled by the admin or not. If you want to remove the gurads after the vlan is removed, you need to use the switch command line interface to do so. Please refer to the documents for the switch command line interfaces for details. + +Limitation +---------- + +Current xCAT-vlan package does not work on the following os distributions. More work will be done in the future releases. + +* ubuntu +* rhel7 and later +* sles12 and later +