Topic: Large number of file in Apache

[gwei]

Hello everyone.
We are using apache for file server and have a large number of files to service. Total 317461 files (Size: about 871G) in the directory which will be shown in the apache. From the 'top' command I see that the apache eats up a lot of VIRT memory(2273m) for every request, and the response of apache is quite slow.

output of 'top' command
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5215 www-data 20 0 2273m 4532 1892 S 5 0.2 0:35.11 apache2

I am not sure if there is any limitation of apache to server large number of file, or if there is anything else that i can do. Hope that you can give me some suggestions to reduce the cost of memory.

Here is the the environment.
$ apache2 -v
Server version: Apache/2.2.16 (Ubuntu)
Server built: Feb 14 2012 18:04:37
$ apache2 -l
Compiled in modules:
core.c
mod_log_config.c
mod_logio.c
worker.c
http_core.c
mod_so.c

Thanks!

[James Blond]

With the default settings apache 2.2.x loads the files in memory before delivering. If the memory is full the OS will put that in the swap.
For me it worked to disable memory mapping --> EnableMMAP Off.

For me it helped also to disable SendFileSupport since on most of my kernels the sendfile implementation is quit buggy.

Try which combination fits best for you.

[gwei]

Thanks for your reply, James.
Seems I found another problem in my apache - it is the LDAP auth which contains some expired username and eats up the server memory. I changed the ldap-user to ldap-group, and the memory usage reduced a lot (from 2273m to 225m).

Your advice is very helpful, I will follow it to optimize my system. Thank you in advance.

[James Blond]

You could also install htop. it is better to monitor the processes than top.

[gwei]

Yes, it is very cool.
thank you.