diff --git a/docs/source/advanced/performance_tuning/httpd_tuning.rst b/docs/source/advanced/performance_tuning/httpd_tuning.rst new file mode 100644 index 000000000..867fdb036 --- /dev/null +++ b/docs/source/advanced/performance_tuning/httpd_tuning.rst @@ -0,0 +1,57 @@ +Tuning httpd for xCAT node deployments +====================================== + +In xCAT, the Operation System provisioning over network is heavily relying on the web server (Apache 2.x). However, Apache 2.x is a general-purpose web server, the default settings may not allow enough simultaneous HTTP client connections to support a large cluster. + + +#. Tuning MaxRequestWorkers directive + +By default, httpd is configured to use ``prefork`` module for **MPM**, which has a limit of 256 simultaneous requests. If any slow httpd response issue was hit during OS provisioning, you can increase **MaxRequestWorkers** directive for greater performance. + +For example, to avoid some nodes provisioning failure when rebooting all nodes in a large hierarchy stateless cluster ( one service node is serving 270 compute nodes ). It is suggested to increased the value from 256 to 1000. + + On Red Hat, change (or add) these directives in + :: + + /etc/httpd/conf/httpd.conf + + + On SLES (with Apache2), change (or add) these directives in + :: + + /etc/apache2/server-tuning.conf + + +#. Having httpd Cache the Files It Is Serving + +Note: this information was contributed by Jonathan Dye and is provided here as an example. The details may have to be changed for distro or apache version. + +This is simplest if you set noderes.nfsserver to a separate apache server, and then you can configure it to reverse proxy and cache. For some reason mod_mem_cache doesn't seem to behave as expected, so you can use mod_disk_cache to achieve a similar result: make a tmpfs on the apache server and configure its mountpoint to be the directory that CacheRoot points to. Also tell it to ignore /install/autoinst since the caching settings are really aggressive. Do a recursive wget to warm the cache and watch the tmpfs fill up. Then do a bunch of kickstart installs. Before this, the apache server on the xcat management node may have been a bottleneck during kickstart installs. After this change, it no longer should be. + +Here is the apache config file: +:: + + ProxyRequests Off # don't be a proxy, just allow the reverse proxy + + CacheIgnoreCacheControl On + CacheStoreNoStore On + CacheIgnoreNoLastMod On + + CacheRoot /var/cache/apache2/tmpfs + CacheEnable disk /install + CacheDisable /install/autoinst + CacheMaxFileSize 1073741824 + + # CacheEnable mem / # failed attempt to do in-memory caching + # MCacheSize 20971520 + # MCacheMaxObjectSize 524288000 + + # through ethernet network + # ProxyPass /install http://172.21.254.201/install + + # through IB network + ProxyPass /install http://192.168.111.2/install + + +For more Apache 2.x tuning, see the external web page: `Apache Performance Tuning `_ + diff --git a/docs/source/advanced/performance_tuning/index.rst b/docs/source/advanced/performance_tuning/index.rst index 30a4fc635..50cd0b2c8 100644 --- a/docs/source/advanced/performance_tuning/index.rst +++ b/docs/source/advanced/performance_tuning/index.rst @@ -10,4 +10,5 @@ The information in this document should be viewed as example data only. Many of linux_os_tuning.rst xcatd_tuning.rst - database_tuning.rst \ No newline at end of file + database_tuning.rst + httpd_tuning.rst diff --git a/docs/source/advanced/performance_tuning/xcatd_tuning.rst b/docs/source/advanced/performance_tuning/xcatd_tuning.rst index 577038326..571639f5c 100644 --- a/docs/source/advanced/performance_tuning/xcatd_tuning.rst +++ b/docs/source/advanced/performance_tuning/xcatd_tuning.rst @@ -22,4 +22,4 @@ For large clusters, you consider changing the default settings in ``site`` table **useflowcontrol** : If ``yes``, the postscript processing on each node contacts ``xcatd`` on the MN/SN using a lightweight UDP packet to wait until ``xcatd`` is ready to handle the requests associated with postscripts. This prevents deploying nodes from flooding ``xcatd`` and locking out admin interactive use. This value works with the **xcatmaxconnections** and **xcatmaxbatch** attributes. If the value is ``no``, nodes sleep for a random time before contacting ``xcatd``, and retry. The default is ``no``. Not supported on AIX. -These attributes may be changed based on the size of your cluster. For a large cluster, it is better to enable **useflowcontrol** and set ``xcatmaxconnection = 128``, ``xcatmaxbatchconnections = 100``. Then the daemon will only allow 100 concurrent connections from the nodes. This will allow 28 connections still to be available on the management node for xCAT commands (e.g ``nodels``). +These attributes may be changed based on the size of your cluster. For a large cluster, it is better to enable **useflowcontrol** and set ``xcatmaxconnection = 356``, ``xcatmaxbatchconnections = 300``. Then the daemon will only allow 300 concurrent connections from the nodes. This will allow 56 connections still to be available on the management node for xCAT commands (e.g ``nodels``).