Author |
|
asafsteier
Joined: 10 Apr 2022 Posts: 7 Location: Tel Aviv, Israel
|
Posted: Sun 10 Apr '22 11:42 Post subject: Apache Httpd process crashes |
|
|
Hi,
I'm using httpd 2.4.52 and openssl 1.1.1m on windows.
Apache crashes and I get the following error in the apache log:
AH00428: Parent: child process exited with status 3221226356 -- Restarting
It seems to be happening at times when child process exists (graceful restart on max connections) and there's a number of websocket connections that are present at that time. MaxConnectionsPerChild is set to 2M.
Backtrace:
mod_proxy.so!connection_cleanup(void * theconn) Line 1555 C
mod_proxy.so!ap_proxy_release_connection(const char * proxy_function, proxy_conn_rec * conn, server_rec * s) Line 2551 C
[Inline Frame] mod_proxy_http.so!ap_proxy_http_cleanup(const char *) Line 1807 C
mod_proxy_http.so!proxy_http_handler(request_rec * r, proxy_worker * worker, proxy_server_conf * conf, char * url, const char * proxyname, unsigned short proxyport) Line 2049 C
mod_proxy.so!proxy_run_scheme_handler(request_rec * r, proxy_worker * worker, proxy_server_conf * conf, char * url, const char * proxyhost, unsigned short proxyport) Line 3432 C
mod_proxy.so!proxy_handler(request_rec * r) Line 1507 C
libhttpd.dll!ap_run_handler(request_rec * r) Line 169 C
libhttpd.dll!ap_invoke_handler(request_rec * r) Line 445 C
libhttpd.dll!ap_process_async_request(request_rec * r) Line 452 C
libhttpd.dll!ap_process_request(request_rec * r) Line 489 C
libhttpd.dll!ap_process_http_sync_connection(conn_rec * c) Line 219 C
libhttpd.dll!ap_run_process_connection(conn_rec * c) Line 42 C
libhttpd.dll!ap_process_connection(conn_rec * c, void * csd) Line 219 C
libhttpd.dll!worker_main(void * thread_num_val) Line 835 C
kernel32.dll!BaseThreadInitThunk () Unknown
ntdll.dll!00007ffd803f1791() Unknown
Has anyone encountered a similar issue? Any ideas how to resolve?
Thanks! |
|
Back to top |
|
Otomatic
Joined: 01 Sep 2011 Posts: 212 Location: Paris, France, EU
|
Posted: Sun 10 Apr '22 14:20 Post subject: |
|
|
In the httpd.conf file, is there :
Code: | # AcceptFilter: On Windows, none uses accept() rather than AcceptEx() and
# will not recycle sockets between connections. This is useful for network
# adapters with broken driver support, as well as some virtual network
# providers such as vpn drivers, or spam, virus or spyware filters.
AcceptFilter http none
AcceptFilter https none
# ThreadStackSize: sets the size of the stack (for autodata)
# of threads which handle client connections and call modules to help process
# those connections. In most cases the operating system default for stack size
# is reasonable, but there are some conditions where it may need to be adjusted.
# Apache httpd may crash when using some third-party modules which use a
# relatively large amount of autodata storage or automatically restart with
# message like: child process 12345 exited with status 3221225725 -- Restarting.
# This type of crash is resolved by setting ThreadStackSize to a value higher
# than the operating system default.
ThreadStackSize 8388608 |
|
|
Back to top |
|
asafsteier
Joined: 10 Apr 2022 Posts: 7 Location: Tel Aviv, Israel
|
Posted: Mon 11 Apr '22 10:16 Post subject: |
|
|
Here's the setting that I have:
<IfModule mpm_winnt_module>
ThreadStackSize 500000
</IfModule>
It did not make a difference.
Also, notice the exist code is different - im getting heap corruption |
|
Back to top |
|
James Blond Moderator
Joined: 19 Jan 2006 Posts: 7373 Location: Germany, Next to Hamburg
|
Posted: Mon 11 Apr '22 11:32 Post subject: |
|
|
The exit code 0xC0000374 is the value returned when heap corruption is detected.
There is often that issue when running PHP as a module in apache.
You can use the forum search and look for 3221226356 |
|
Back to top |
|
asafsteier
Joined: 10 Apr 2022 Posts: 7 Location: Tel Aviv, Israel
|
Posted: Tue 12 Apr '22 13:46 Post subject: |
|
|
Thanks. I don't use PHP. I've searched the forum but could not find anything relevant |
|
Back to top |
|
asafsteier
Joined: 10 Apr 2022 Posts: 7 Location: Tel Aviv, Israel
|
Posted: Wed 13 Apr '22 14:28 Post subject: |
|
|
Another thing to mention: we noticed that when the child process is exiting, the crash comes from trying to clean a pool resource (e.g. a connection pool) that was already destroyed before as a decendant of the main process pool |
|
Back to top |
|
James Blond Moderator
Joined: 19 Jan 2006 Posts: 7373 Location: Germany, Next to Hamburg
|
Posted: Tue 03 May '22 8:34 Post subject: |
|
|
Do you set the following as Otomatic suggested?
AcceptFilter http none
AcceptFilter https none |
|
Back to top |
|
asafsteier
Joined: 10 Apr 2022 Posts: 7 Location: Tel Aviv, Israel
|
Posted: Tue 03 May '22 14:32 Post subject: |
|
|
Thanks JB, yes we already have those |
|
Back to top |
|
James Blond Moderator
Joined: 19 Jan 2006 Posts: 7373 Location: Germany, Next to Hamburg
|
Posted: Wed 04 May '22 8:39 Post subject: |
|
|
I would give it a try to increase ThreadStackSizeby a lot.
Your value is very low.
0500000 your value
8388608 default value.
The default 8388608 is only 8 MiB.
or increase the number of Threads Per Child. The default of 64 for windows can be too low. On my productive systems, I use 150
Code: |
ThreadsPerChild 150
|
3221226356 aka heap corruption is that the buffer is too small. |
|
Back to top |
|