Operations/Server Documentation/Resource Analysis
From Habari Project
This page is outdated an may not be relevant to the current version of Habari. You can help Habari by updating it.
This page was created based on the old Habari server and has not been completely updated with information from the new box.
Contents |
Abstract
Due to the un-funded nature of the Habari project and the wide range of services offered under the habariproject.org domain, efficient utilization of server resources is a priority. To that end significant monitoring and research has been done to best characterize the typical working environment available on the hp.o server.
Data Gathering
Before making any assumptions we need data. While memory usage fluctuates greatly there are baselines that have been established over a significant monitoring interval. Both average and extreme usage is illustrated below. For analysis purposes we are only looking at the Resident Set Size (RSS), which should exclude things like SWAP usage from the calculations.
Average
The below snippet [1] portrays a good estimate of the average system load.
# ps -Aly --sort:rss S UID PID PPID C PRI NI RSS SZ WCHAN TTY TIME CMD ... S 33 32348 2881 0 80 0 4036 18770 - ? 00:05:46 nginx S 33 32347 2881 0 80 0 4620 18849 - ? 00:05:44 nginx S 33 32346 2881 0 80 0 4872 18928 - ? 00:05:48 nginx S 33 32345 2881 0 80 0 4968 18898 - ? 00:05:40 nginx S 33 1209 21562 0 80 0 25388 93509 - ? 00:00:19 apache2 S 103 2363 1 1 80 0 43228 78957 - ? 06:30:32 mysqld S 33 12477 2694 1 80 0 69228 62492 - ? 00:52:32 php5-fpm S 33 12476 2694 1 80 0 69308 62449 - ? 00:52:23 php5-fpm S 33 2935 2694 0 80 0 69800 62850 - ? 03:58:39 php5-fpm S 33 2695 2694 0 80 0 71080 62711 - ? 03:57:32 php5-fpm S 33 2886 2694 0 80 0 77708 63618 - ? 03:58:08 php5-fpm S 33 2845 21562 0 80 0 103924 117171 - ? 00:04:06 apache2 S 33 15958 1 17 80 0 114684 100944 - ? 03:36:18 tracd
Extreme
In some instances the memory usage of individual Apache and Trac processes can balloon to significantly higher levels, as illustrated in the below snippet [2]:
# ps -Aly --sort:rss S UID PID PPID C PRI NI RSS SZ WCHAN TTY TIME CMD ... S 33 11554 6275 0 80 0 10484 78622 semtim ? 00:00:00 apache2 S 33 11437 6275 0 80 0 12048 78690 semtim ? 00:00:00 apache2 S 0 6275 1 0 80 0 15920 78473 d_look ? 00:00:00 apache2 S 33 11158 6275 0 80 0 21580 79239 semtim ? 00:00:00 apache2 S 33 11397 6275 0 80 0 36056 79854 semtim ? 00:00:00 apache2 S 33 7326 6276 0 80 0 41940 61848 - ? 00:00:16 trac.fcgi S 33 7324 6276 1 80 0 46500 64761 - ? 00:00:21 trac.fcgi S 33 7325 6276 1 80 0 47504 81228 - ? 00:00:24 trac.fcgi S 33 11350 6275 0 80 0 58588 88353 - ? 00:00:00 apache2 S 33 6317 6276 2 80 0 70028 70428 - ? 00:01:04 trac.fcgi S 106 7182 7143 0 80 0 113976 85284 - ? 00:00:16 mysqld
Analysis
The top consumers of memory are MySQL, Apache, and Trac. Each has a baseline defined.
MySQL
Virtually all of the services provided on hp.o are database-based, so the highest memory consumption on the box is logically the MySQL process. The caching values of MySQL have been tuned extensively for the specific use cases involved and are detailed in MySQL Configuration.
In general the assumption is that MySQL will regularly use at least 128M of RAM for caching and up to an additional 40M for combined other temporary sorting of results, etc. Total, it can be expected to easily approach 180M during load.
Apache
Behind MySQL is the web server itself, Apache. The main variable in an mpm_prefork Apache environment is the number of processes allowed.
For our purposes Apache can easily consume over 50 - 55M of RAM per process.
Trac
Trac's memory usage varies more widely than anything else on the system, making it exceptionally difficult to predict.
Each Trac process should be expected to easily consume 35 - 40M. As seen in the Extreme example, Trac should be afforded extra care and anticipated to use up to more than twice this amount.
