Keep Server Online
If you find the Apache Lounge, the downloads and overall help useful, please express your satisfaction with a donation.
or
A donation makes a contribution towards the costs, the time and effort that's going in this site and building.
Thank You! Steffen
Your donations will help to keep this site alive and well, and continuing building binaries. Apache Lounge is not sponsored.
| |
|
 Topic: Apache runs out of threads but we're maxed out |
|
| Author |
|
lemonthyme
Joined: 29 Aug 2019
Posts: 2
|
 Posted: Thu 29 Aug '19 14:38 Post subject: Apache runs out of threads but we're maxed out |
 |
|
Background
we inherited maintenance of an old application at a training organization. It's hosted on a Windows 10 machine with an SSD and 4GB of RAM. Trainees each have a QR code which they scan in the training labs to verify their presence. Multiple training sessions are going on simultaneously and most also finish at the same time meaning that there is a large number of trainees scanning their QR codes at the same time.
The Problem
On some occasions, the application stops responding and does not read QR codes. The host machine lags to the point of being unusable forcing a restart to get the application running again. Apache's error log says "Server ran out of threads to serve requests. Consider raising the ThreadsPerChild setting" which is already maxed out at 1920 threads (a warning is printed to the Apache error log if we try to exceed 1920)
Sometimes when this happens, the following error is also printed to the Apache error log: "(OS 64)The specified network name is no longer available. : winnt_accept: Asynchronous AcceptEx failed."
We have tried
1. Raising ThreadsPerChild. As mentioned, it's currently at 1920 and can't be raised any more.
2. Turning off KeepAlive. Feedback from users seem to indicate this made things worse so we turned it back on.
System Information
Host machine is running Windows 10 and runs XAMPP 1.7.3 which provides Apache 2.2.14, MySQL 5.1.41 & PHP 5.3.1. The application itself was coded in PHP.
Our Options
Due to the nature of the application, our first choice would be to tune or upgrade Apache (in that order). We are unsure if the application will run on PHP 7 or greater and to verify would mean a great deal of testing (and there's no guarantee it would solve the issue). A suggestion was made to switch to NGINX but some rudimentary testing shows that the application makes heavy use of Apache URL re-writes and converting them into NGINX Directives would be overly complicated.
A last resort would be to recode the portion of the application that does the QR scanning. It's possible but we'd like to keep it to the last resort.
We've been scouring the web for possible solutions but haven't turned up anything yet. We're hoping someone here could help shed some light. |
|
| Back to top |
|
James Blond
Moderator
Joined: 19 Jan 2006
Posts: 7371
Location: Germany, Next to Hamburg
|
| Posted: Thu 29 Aug '19 14:55 Post subject: Re: Apache runs out of threads but we're maxed out |
|
|
| lemonthyme wrote: | [
Sometimes when this happens, the following error is also printed to the Apache error log: "(OS 64)The specified network name is no longer available. : winnt_accept: Asynchronous AcceptEx failed."
|
With that error add the following to your httpd.conf file and restart apache
| Code: |
AcceptFilter https none
AcceptFilter http none
EnableSendfile Off
EnableMMAP off
|
|
|
| Back to top |
|
glsmith
Moderator
Joined: 16 Oct 2007
Posts: 2268
Location: Sun Diego, USA
|
| Posted: Fri 30 Aug '19 0:32 Post subject: |
|
|
| lemonthyme wrote: |
System Information
Host machine is running Windows 10 and runs XAMPP 1.7.3 which provides Apache 2.2.14, MySQL 5.1.41 & PHP 5.3.1. The application itself was coded in PHP.
|
Just pointing out the fact to others that you're on 2.2, not 2.4.
What about 2.2's Win32DisableAcceptEx? After all, AcceptEx() is causing a problem, but not the root problem of running out of threads.
http://httpd.apache.org/docs/2.2/mod/mpm_winnt.html#win32disableacceptex
AcceptFilter https none
AcceptFilter http none
will do exactly the same, just two lines instead of one.
<soapbox>
I have a problem with this specific copy and paste reply I see a lot to the AccpetEx() problem because EnableSendfile & EnableMMAP have absolutely nothing to do with AcceptEx() and does have other real world pros & cons that can become beneficial or problematic. Mileage will vary per system.
Also note that EnableSendfile's default changed in 2.3.9.
http://httpd.apache.org/docs/2.2/mod/core.html#enablesendfile
http://httpd.apache.org/docs/2.4/mod/core.html#enablesendfile
EnableSendfile On saves RAM but at a cost to speed, probably not so much with an SSD, yet it is 2.2's default so changing this will change Apache's profile. With 4GB of RAM this will likely become noticeable on file transfers over 800MB+/- finishing incomplete. I find the only way I can have successful large file transfers (with 4GB ram) is to put them in a folder that I can specifically turn EnableSendfile to On for or to use <Files> instead. I also find that small files do much better with it turned Off.
This is probably the one change that really sped up 2.4. The ram hit to it was likely washed away by modulizing darn near everything but the really core features. No need for it, don't load it, ram saved.
Turning EnableMMAP to off also has nothing to do with AcceptEx(). I do not see anything that would make me think this is bad idea, because it saves RAM as I read it, but one should understand what it does, as best as they can anyway by reading about it before they just go and do it. It could become problematic on configurations that use the Includes option, like mine. I have never turned it off, so I don't know if I'd benefit from it or not.
http://httpd.apache.org/docs/2.2/mod/core.html#enablemmap
</soapbox>
Edit: How any of these have to do with threads I am not sure. Other than possibly each thread require x amount of ram (seems logical) I do not know. If anyone does, please enlighten me.
Reducing unnecessary ram use I think is never a bad idea because every server has it's limits, you just may not have enough server to run so many concurrent connections. |
|
| Back to top |
|
lemonthyme
Joined: 29 Aug 2019
Posts: 2
|
| Posted: Wed 04 Sep '19 10:31 Post subject: |
|
|
Thanks to all who have replied so far.
| Quote: |
AcceptFilter https none
AcceptFilter http none
EnableSendfile Off
EnableMMAP off
|
The 2 AcceptFilter commands are already in our httpd.conf file. We will try adding the commands to turn off the EnableSendfile and EnableMMAP.
Will also give Win32DisableAcceptEx a shot. If this doesn't work we will probably look into upgrading Apache. |
|
| Back to top |
|
|
|
|
|
|
