From edfedde7866fe0829881ac40e193e549d9e5932b Mon Sep 17 00:00:00 2001 From: penguhyang Date: Mon, 14 Mar 2016 04:33:47 -0400 Subject: [PATCH] update xcatdebugmode document in readthedocs --- docs/source/index.rst | 1 + docs/source/troubleshooting/index.rst | 10 +++ .../os_installation/debug_port.rst | 21 ++++++ .../troubleshooting/os_installation/index.rst | 44 ++++++++++++ .../os_installation/log_to_mn_cn.rst | 51 ++++++++++++++ .../os_installation/ssh_enable.rst | 67 +++++++++++++++++++ 6 files changed, 194 insertions(+) create mode 100644 docs/source/troubleshooting/index.rst create mode 100644 docs/source/troubleshooting/os_installation/debug_port.rst create mode 100644 docs/source/troubleshooting/os_installation/index.rst create mode 100644 docs/source/troubleshooting/os_installation/log_to_mn_cn.rst create mode 100644 docs/source/troubleshooting/os_installation/ssh_enable.rst diff --git a/docs/source/index.rst b/docs/source/index.rst index 5dc098fca..242827176 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -32,6 +32,7 @@ Table of Contents guides/install-guides/index.rst guides/admin-guides/index.rst advanced/index.rst + troubleshooting/index.rst developers/index.rst help.rst diff --git a/docs/source/troubleshooting/index.rst b/docs/source/troubleshooting/index.rst new file mode 100644 index 000000000..5d1f1124c --- /dev/null +++ b/docs/source/troubleshooting/index.rst @@ -0,0 +1,10 @@ +Troubleshooting +=============== + +This chapter introduces the methods of how to troubleshoot a real cluster. Basically, it includes the following parts. + +.. toctree:: + :maxdepth: 2 + + os_installation/index.rst + diff --git a/docs/source/troubleshooting/os_installation/debug_port.rst b/docs/source/troubleshooting/os_installation/debug_port.rst new file mode 100644 index 000000000..c80d891be --- /dev/null +++ b/docs/source/troubleshooting/os_installation/debug_port.rst @@ -0,0 +1,21 @@ +Enable Debug Port: Running commands in the installer from MN +------------------------------------------------------------ + +**This mode is only supported with debug level set to 1 or 2** + +xCAT creates a server in the **installer** listening on port ``3001``. It executes commands sent to it from the xCAT MN and returns the response output. + +The command ``runcmdinstaller`` can be used to send request to installer: :: + + Usage: + + runcmdinstaller "" + + make sure all the commands are quoted by "" + + Example: + + To list all the items under the /etc directory in the installer + + runcmdinstaller c910f03c01p03 "ls /etc" + diff --git a/docs/source/troubleshooting/os_installation/index.rst b/docs/source/troubleshooting/os_installation/index.rst new file mode 100644 index 000000000..dfdff4a0c --- /dev/null +++ b/docs/source/troubleshooting/os_installation/index.rst @@ -0,0 +1,44 @@ +Operating System Installation +============================= + +The ability to access the installer or to collect logs during the installation process can be helpful when debugging installation problems. + +A new attribute is provided in the site table called ``xcatdebugmode``. :: + + xcatdebugmode=0: Only diagnose Log will be show in corresponding files. + xcatdebugmode=1: Diagnose Log will be show in corresponding files and debug port will be opened. + xcatdebugmode=2: SSH is supported while installing also with diagnose log show and debug port enable. + +List of Supported OS. :: + + RHEL: 6.7 and above + SLES: 11.4 and above + UBT: 14.04.3 and above + +The following behavior is observed during OS install: + ++-----------------+--------------+--------------+--------------+ +|**xcatdebugmode**| 0 | 1 | 2 | ++-----------------+----+----+----+----+----+----+----+----+----+ +| |RHEL|SLES|UBT |RHEL|SLES|UBT |RHEL|SLES|UBT | ++=================+====+====+====+====+====+====+====+====+====+ +| Log Collecting | Y | Y | Y | Y | Y | Y | Y | Y | Y | ++-----------------+----+----+----+----+----+----+----+----+----+ +|Enable Debug Port| N | N | N | Y | Y | Y | Y | Y | Y | ++-----------------+----+----+----+----+----+----+----+----+----+ +| SSH Access | N | N | N | N | N | N | Y | Y | Y | ++-----------------+----+----+----+----+----+----+----+----+----+ + +Y means the behavior is supported by OS at current xcatdebugmode. + +N means the opposite meaning. + +This chapter introduces the procedures of how to troubleshoot operating system installation. Basically, it includes the following parts. + +.. toctree:: + :maxdepth: 2 + + log_to_mn_cn.rst + debug_port.rst + ssh_enable.rst + diff --git a/docs/source/troubleshooting/os_installation/log_to_mn_cn.rst b/docs/source/troubleshooting/os_installation/log_to_mn_cn.rst new file mode 100644 index 000000000..8b19712b3 --- /dev/null +++ b/docs/source/troubleshooting/os_installation/log_to_mn_cn.rst @@ -0,0 +1,51 @@ +Log Collecting: Collecting logs of the whole installation process +----------------------------------------------------------------- + +The ability to collect logs during the installation process can be helpful when debugging installation problems. + +Pre-Install: the logs of pre-installation scripts, the pre-installation scripts include "%pre" section in anaconda, "" section for SUSE and "partman/early_command" and "preseed/early_command" sections for ubuntu. The logs include the STDOUT and STDERR of the scripts as well as the debug trace output of bash scripts with "set -x" + +Installer: the logs from the os installer itself, i.e, the logs of installation program(anaconda, autoyast and preseed,etc.) + +Post-Install: the logs of post-installation scripts, the post-installation scripts include "%post" section in anaconda, "" and "" sections for SUSE and "preseed/late_command" section for ubuntu. The logs include the STDOUT and STDERR of the scripts as well as the debug trace output of bash scripts with "set -x" + +Post-Script: This section is useful for functions such as updating node status. This section contains the postbootscripts. + +The following behavior is observed during OS install: + ++------------------+--------------+--------------+--------------+ +|**xcatdebugmode** | 0 | 1 | 2 | ++------------------+----+----+----+----+----+----+----+----+----+ +| |RHEL|SLES|UBT |RHEL|SLES|UBT |RHEL|SLES|UBT | ++=============+====+====+====+====+====+====+====+====+====+====+ +| Pre-Install | MN | N | N | N | ++ log +----+----+----+----+----+----+----+----+----+----+ +| | CN | Y1 | Y2 | Y2 | ++-------------+----+----+----+----+----+----+----+----+----+----+ +| Installer | MN | N | N | N | Y6 | Y6 | N | Y6 | Y6 | N | ++ log +----+----+----+----+----+----+----+----+----+----+ +| | CN | Y5 | Y5 | Y5 | Y5 | Y5 | Y5 | Y5 | Y5 | Y5 | ++-------------+----+----+----+----+----+----+----+----+----+----+ +| Post-Install| MN | Y4 | Y3 | Y3 | ++ log +----+----+----+----+----+----+----+----+----+----+ +| | CN | Y1 | Y2 | Y2 | ++-------------+----+----+----+----+----+----+----+----+----+----+ +| Post-Script | MN | Y4 | Y3 | Y3 | ++ log +----+----+----+----+----+----+----+----+----+----+ +| | CN | Y1 | Y2 | Y2 | ++-------------+----+----+----+----+----+----+----+----+----+----+ + +Y1 means the installation logs can be saved to ``/var/log/xcat/xcat.log`` file on CN. + +Y2 means the installation logs and debug trace("set -x" or "-o xtrace") of bash scripts can be saved to ``/var/log/xcat/xcat.log`` file on CN. + +Y3 means the installation logs can be forwarded to ``/var/log/xcat/computes.log`` file on MN. + +Y4 means the error messages can be forwarded to ``/var/log/xcat/computes.log`` file on MN only when critical error happens. + +Y5 means the installer log can be saved to the CN in ``/var/log/anaconda`` for RHEL, ``/var/log/YaST2`` for SLES, ``/var/log/installer`` for UBT. + +Y6 means the installer log can be forwarded to the MN in ``/var/log/xcat/computes.log`` file. + +N means the logs can not be forwarded or saved. + diff --git a/docs/source/troubleshooting/os_installation/ssh_enable.rst b/docs/source/troubleshooting/os_installation/ssh_enable.rst new file mode 100644 index 000000000..13f09dd29 --- /dev/null +++ b/docs/source/troubleshooting/os_installation/ssh_enable.rst @@ -0,0 +1,67 @@ +SSH Access: Accessing the installer via "ssh" +--------------------------------------------- + +**This mode is only supported with debug level set to 2** + +The ssh access to the installer is enabled, the admin can login into the installer through: + +#. For RHEL, the installation won't halt, just login into the installer with ``ssh root@``. + +#. For SLES, the installation will halt after the ssh server is started, the console output looks like: :: + + *** sshd has been started *** + + + *** login using 'ssh -X root@' *** + *** run 'yast' to start the installation *** + + Just as the message above suggests, the admin can open 2 sessions and run ``ssh -X root@`` with the configured system password in the ``passwd`` table to login into the installer, then run ``yast`` to continue installation in one session and inspect the installation process in the installer in the other session. + + After the installation is finished, the system requires a reboot. The installation will halt again before the system configuration, the console output looks like: :: + + *** Preparing SSH installation for reboot *** + *** NOTE: after reboot, you have to reconnect and call yast.ssh *** + + Just as the message above suggests, the admin should run ``ssh -X root@`` to access the installer and run ``yast.ssh`` to finish the installation. + +#. For UBT, the installation will halt on the following similar message in the console: :: + + ┌───────────┤ [!!] Continue installation remotely using SSH ├───────────┐ + │ │ + │ Start SSH │ + │ To continue the installation, please use an SSH client to connect to │ + │ the IP address and log in as the "installer" user. For │ + │ example: │ + │ │ + │ ssh installer@ │ + │ │ + │ The fingerprint of this SSH server's host key is: │ + │ │ + │ │ + │ Please check this carefully against the fingerprint reported by your │ + │ SSH client. │ + │ │ + │ │ + │ │ + └───────────────────────────────────────────────────────────────────────┘ + + Just as the message show, the admin can run ``ssh installer@`` with the password "cluster" to login into the installer, the following message shows on login: :: + + ┌────────────────────┤ [!!] Configuring d-i ├─────────────────────┐ + │ │ + │ This is the network console for the Debian installer. From │ + │ here, you may start the Debian installer, or execute an │ + │ interactive shell. │ + │ │ + │ To return to this menu, you will need to log in again. │ + │ │ + │ Network console option: │ + │ │ + │ Start installer │ + │ Start installer (expert mode) │ + │ Start shell │ + │ │ + └─────────────────────────────────────────────────────────────────┘ + + The admin can open 2 sessions and then select "Start installer" to continue installation in one session and select "Start shell" in the other session to inspect the installation process in the installer. +