logo
Apache Lounge
Webmasters

 

About Forum Index Downloads Search Register Log in RSS X


Keep Server Online

If you find the Apache Lounge, the downloads and overall help useful, please express your satisfaction with a donation.

or

Bitcoin

A donation makes a contribution towards the costs, the time and effort that's going in this site and building.

Thank You! Steffen

Your donations will help to keep this site alive and well, and continuing building binaries. Apache Lounge is not sponsored.
Post new topic   Forum Index -> Apache View previous topic :: View next topic
Reply to topic   Topic: Apache runs unstable, keeps restarting in an erratic way
Author
chrisiwien



Joined: 14 May 2024
Posts: 17
Location: Austria, Vienna

PostPosted: Thu 16 May '24 16:38    Post subject: Apache runs unstable, keeps restarting in an erratic way Reply with quote

Hello,

I am facing serious stability problems with Apache for Windows on a Windows Server 2022. I am currently running two different websites on the server, both websites are WordPress sites with WooCommerce. My customer's requirement is that windows programs (exe and bat files) must run on the server. This files return values via php. These Windows executables are not called when the websites are loaded, but are only used when specific user input is made.

-------------------------------------
FIRST PROBLEM step by step explained:
-------------------------------------
- I start the Apache Server
- I try to reach website A in the webbrowser
- The first call of website A takes very long, around 10 to 20 seconds.
- Then, i can browse quickly this website
- When i call website B the FIRST TIME AFTER APACHE-RESTART, website B takes also 10 to 20 seconds to load.
- If i browse meanwhile on website A during this time, this website is also unreachable - and produce an error message in the browser.
- During this procedure, several error log entries are created in the Apache error log.

What is happening?

As far, as i understand, Apache for Windows works differently than the Linux version, only one "master" child process is created, and this process creates further sub processes. But why affect the call of one website the call of another website on the same server? That can not be right. Are all website requests handled by the "master" child process or by the sub processes?


============================================================
The Server hardware specs:
CPU: i5-13500 with 14 cores
RAM: 64 GB RAM
Storage: SSD

Services installed and running on the server:
Apache for Windows version: Apache/2.4.58 (Win64), Apache Lounge VS17 Server built: Jan 30 2024 19:01:23 (I have turned LogLevel in httpd.conf to debug)
MySQL-Server version: 8.0.36
PHP-Version: 8.1.21 Thread safe as a module with php8apache2_4.dll
- max_input_vars = 5000
- memory_limit = 512M
- post_max_size = 256M
- upload_max_filesize = 256M

============================================================

-------------------------------
SECOND (maybe related) PROBLEM:
-------------------------------
The Apache server keeps exiting child processes and restarting in an erratic behaviour, at any time.


Messages in the Apache error log:

AH00422: Parent: Received shutdown signal -- Shutting down the server.

AH00428: Parent: child process 4700 exited with status 3221225477
I follwed the recommendation to set the value of "ThreadStackSize 8888888" in httpd-mpm.conf
-> no improvement

I tried increasing the value of "ThreadsPerChild" 10000 in httpd-mpm.conf - but Apache doesn't allow that on Windows, ThreadPerChild are limited to 1920. Why is that, and how can I raise this value in Apache for Windows? Would increasing this value improve stability?

What is happing there? The server hardware is rather powerful, so there shouldn't be these problems. What can I do to solve this problems, so the Apache for Windows webserver runs stable?





============================================================
Further log entries during the exit child process/restarting child process procedure:

[Thu May 16 02:12:31.604125 2024] [mpm_winnt:notice] [pid 3184:tid 328] AH00418: Parent: Created child process 4700
[Thu May 16 02:12:31.775982 2024] [security2:notice] [pid 4700:tid 364] ModSecurity for Apache/2.9.7 (http://www.modsecurity.org/) configured.
[Thu May 16 02:12:31.775982 2024] [security2:notice] [pid 4700:tid 364] ModSecurity: APR compiled version="1.7.0"; loaded version="1.7.4"
[Thu May 16 02:12:31.775982 2024] [security2:warn] [pid 4700:tid 364] ModSecurity: Loaded APR do not match with compiled!
[Thu May 16 02:12:31.775982 2024] [security2:notice] [pid 4700:tid 364] ModSecurity: PCRE2 compiled version="10.40 "; loaded version="10.42 2022-12-11"
[Thu May 16 02:12:31.775982 2024] [security2:warn] [pid 4700:tid 364] ModSecurity: Loaded PCRE do not match with compiled!
[Thu May 16 02:12:31.775982 2024] [security2:notice] [pid 4700:tid 364] ModSecurity: LUA compiled version="Lua 5.3"
[Thu May 16 02:12:31.775982 2024] [security2:notice] [pid 4700:tid 364] ModSecurity: YAJL compiled version="2.1.0"
[Thu May 16 02:12:31.775982 2024] [security2:notice] [pid 4700:tid 364] ModSecurity: LIBXML compiled version="2.10.3"
[Thu May 16 02:12:31.775982 2024] [security2:notice] [pid 4700:tid 364] ModSecurity: StatusEngine call: "2.9.7,Apache,1.7.0/1.7.4,10.40/,Lua 5.3,2.10.3,754d0799beee818dab333d3913436bf3ee9fd6e3"
[Thu May 16 02:12:31.775982 2024] [security2:notice] [pid 4700:tid 364] ModSecurity: StatusEngine call failed. Query: GIXDSLRXFRAXAYLDNBSSYMJOG4XDAL.ZRFY3S4NBMGEYC4NBQF4WEY5LBEA2S.4MZMGIXDCMBOGMWDONJUMQYDOOJZMJ.SWKZJYGE4GIYLCGMZTGZBTHEYTGNBT.GZRGMM3FMU4WMZBWMUZQ.1715818351.status.modsecurity.org
[Thu May 16 02:12:31.838476 2024] [mpm_winnt:notice] [pid 4700:tid 364] AH00354: Child: Starting 1920 worker threads.
[Thu May 16 14:26:10.490079 2024] [mpm_winnt:notice] [pid 3184:tid 328] AH00428: Parent: child process 4700 exited with status 3221225477 -- Restarting.
[Thu May 16 14:26:11.434048 2024] [mpm_winnt:notice] [pid 3184:tid 328] AH00455: Apache/2.4.58 (Win64) OpenSSL/3.1.5 configured -- resuming normal operations
[Thu May 16 14:26:11.434048 2024] [mpm_winnt:notice] [pid 3184:tid 328] AH00456: Apache Lounge VS17 Server built: Jan 30 2024 19:01:23
[Thu May 16 14:26:11.434048 2024] [core:notice] [pid 3184:tid 328] AH00094: Command line: '{server-root}\\Apache24\\bin\\httpd.exe -d {server-root}/Apache24'


[Thu May 16 14:48:08.723532 2024] [security2:notice] [pid 9808:tid 352] ModSecurity: StatusEngine call failed. Query: GIXDSLRXFRAXAYLDNBSSYMJOG4XDAL.ZRFY3S4NBMGEYC4NBQF4WEY5LBEA2S.4MZMGIXDCMBOGMWDONJUMQYDOOJZMJ.SWKZJYGE4GIYLCGMZTGZBTHEYTGNBT.GZRGMM3FMU4WMZBWMUZQ.1715863677.status.modsecurity.org
[Thu May 16 14:48:08.821789 2024] [ssl:info] [pid 9808:tid 352] AH01883: Init: Initialized OpenSSL library
[Thu May 16 14:48:08.821789 2024] [ssl:debug] [pid 9808:tid 352] ssl_engine_init.c(353): AH01886: OpenSSL has FIPS mode disabled
[Thu May 16 14:48:08.821789 2024] [socache_shmcb:debug] [pid 9808:tid 352] mod_socache_shmcb.c(414): AH00821: shmcb_init allocated 512000 bytes of shared memory
[Thu May 16 14:48:08.821789 2024] [socache_shmcb:debug] [pid 9808:tid 352] mod_socache_shmcb.c(428): AH00822: for 511944 bytes (512000 including header), recommending 32 subcaches, 88 indexes each
[Thu May 16 14:48:08.821789 2024] [socache_shmcb:debug] [pid 9808:tid 352] mod_socache_shmcb.c(465): AH00824: shmcb_init_memory choices follow
[Thu May 16 14:48:08.821789 2024] [socache_shmcb:debug] [pid 9808:tid 352] mod_socache_shmcb.c(467): AH00825: subcache_num = 32
[Thu May 16 14:48:08.821789 2024] [socache_shmcb:debug] [pid 9808:tid 352] mod_socache_shmcb.c(469): AH00826: subcache_size = 15992
[Thu May 16 14:48:08.821789 2024] [socache_shmcb:debug] [pid 9808:tid 352] mod_socache_shmcb.c(471): AH00827: subcache_data_offset = 2128
[Thu May 16 14:48:08.821789 2024] [socache_shmcb:debug] [pid 9808:tid 352] mod_socache_shmcb.c(473): AH00828: subcache_data_size = 13864
[Thu May 16 14:48:08.821789 2024] [socache_shmcb:debug] [pid 9808:tid 352] mod_socache_shmcb.c(475): AH00829: index_num = 88
[Thu May 16 14:48:08.821789 2024] [socache_shmcb:info] [pid 9808:tid 352] AH00830: Shared memory socache initialised
[Thu May 16 14:48:08.821789 2024] [ssl:info] [pid 9808:tid 352] AH01887: Init: Initializing (virtual) servers for SSL
[Thu May 16 14:48:08.834294 2024] [ssl:info] [pid 9808:tid 352] AH01876: mod_ssl/2.4.58 compiled against Server: Apache/2.4.58, Library: OpenSSL/3.1.5
[Thu May 16 14:48:08.834294 2024] [http2:info] [pid 9808:tid 352] AH03090: mod_http2 (v2.0.22, feats=, nghttp2 1.59.0), initializing...
[Thu May 16 14:48:08.865962 2024] [mpm_winnt:notice] [pid 9808:tid 352] AH00455: Apache/2.4.58 (Win64) OpenSSL/3.1.5 configured -- resuming normal operations
[Thu May 16 14:48:08.865962 2024] [mpm_winnt:notice] [pid 9808:tid 352] AH00456: Apache Lounge VS17 Server built: Jan 30 2024 19:01:23
[Thu May 16 14:48:08.865962 2024] [core:notice] [pid 9808:tid 352] AH00094: Command line: '{server-root}\\Apache24\\bin\\httpd.exe -d {server-root}/Apache24'
[Thu May 16 14:48:08.865962 2024] [core:debug] [pid 9808:tid 352] log.c(1597): AH02639: Using SO_REUSEPORT: no (0)
[Thu May 16 14:48:08.865962 2024] [mpm_winnt:notice] [pid 9808:tid 352] AH00418: Parent: Created child process 9980
[Thu May 16 14:48:08.865962 2024] [mpm_winnt:debug] [pid 9808:tid 352] mpm_winnt.c(430): AH00402: Parent: Sent the scoreboard to the child
[Thu May 16 14:48:08.992953 2024] [ssl:info] [pid 9980:tid 344] AH01883: Init: Initialized OpenSSL library
[Thu May 16 14:48:08.992953 2024] [ssl:debug] [pid 9980:tid 344] ssl_engine_init.c(353): AH01886: OpenSSL has FIPS mode disabled
[Thu May 16 14:48:08.992953 2024] [ssl:info] [pid 9980:tid 344] AH01887: Init: Initializing (virtual) servers for SSL
[Thu May 16 14:48:08.992953 2024] [ssl:info] [pid 9980:tid 344] AH01876: mod_ssl/2.4.58 compiled against Server: Apache/2.4.58, Library: OpenSSL/3.1.5
[Thu May 16 14:48:08.992953 2024] [http2:debug] [pid 9980:tid 344] mod_http2.c(112): AH03089: initializing post config dry run
[Thu May 16 14:48:08.992953 2024] [security2:notice] [pid 9980:tid 344] ModSecurity for Apache/2.9.7 (http://www.modsecurity.org/) configured.
[Thu May 16 14:48:08.992953 2024] [security2:notice] [pid 9980:tid 344] ModSecurity: APR compiled version="1.7.0"; loaded version="1.7.4"
[Thu May 16 14:48:08.992953 2024] [security2:warn] [pid 9980:tid 344] ModSecurity: Loaded APR do not match with compiled!
[Thu May 16 14:48:08.992953 2024] [security2:notice] [pid 9980:tid 344] ModSecurity: PCRE2 compiled version="10.40 "; loaded version="10.42 2022-12-11"
[Thu May 16 14:48:08.992953 2024] [security2:warn] [pid 9980:tid 344] ModSecurity: Loaded PCRE do not match with compiled!
[Thu May 16 14:48:08.992953 2024] [security2:notice] [pid 9980:tid 344] ModSecurity: LUA compiled version="Lua 5.3"
[Thu May 16 14:48:08.992953 2024] [security2:notice] [pid 9980:tid 344] ModSecurity: YAJL compiled version="2.1.0"
[Thu May 16 14:48:08.992953 2024] [security2:notice] [pid 9980:tid 344] ModSecurity: LIBXML compiled version="2.10.3"
[Thu May 16 14:48:08.992953 2024] [security2:notice] [pid 9980:tid 344] ModSecurity: StatusEngine call: "2.9.7,Apache,1.7.0/1.7.4,10.40/,Lua 5.3,2.10.3,754d0799beee818dab333d3913436bf3ee9fd6e3"
[Thu May 16 14:48:15.032950 2024] [security2:notice] [pid 9980:tid 344] ModSecurity: StatusEngine call failed. Query: GIXDSLRXFRAXAYLDNBSSYMJOG4XDAL.ZRFY3S4NBMGEYC4NBQF4WEY5LBEA2S.4MZMGIXDCMBOGMWDONJUMQYDOOJZMJ.SWKZJYGE4GIYLCGMZTGZBTHEYTGNBT.GZRGMM3FMU4WMZBWMUZQ.1715863688.status.modsecurity.org
[Thu May 16 14:48:15.080521 2024] [ssl:info] [pid 9980:tid 344] AH01883: Init: Initialized OpenSSL library
[Thu May 16 14:48:15.080521 2024] [ssl:debug] [pid 9980:tid 344] ssl_engine_init.c(353): AH01886: OpenSSL has FIPS mode disabled
[Thu May 16 14:48:15.080521 2024] [socache_shmcb:debug] [pid 9980:tid 344] mod_socache_shmcb.c(414): AH00821: shmcb_init allocated 512000 bytes of shared memory
[Thu May 16 14:48:15.080521 2024] [socache_shmcb:debug] [pid 9980:tid 344] mod_socache_shmcb.c(428): AH00822: for 511944 bytes (512000 including header), recommending 32 subcaches, 88 indexes each
[Thu May 16 14:48:15.080521 2024] [socache_shmcb:debug] [pid 9980:tid 344] mod_socache_shmcb.c(465): AH00824: shmcb_init_memory choices follow
[Thu May 16 14:48:15.080521 2024] [socache_shmcb:debug] [pid 9980:tid 344] mod_socache_shmcb.c(467): AH00825: subcache_num = 32
[Thu May 16 14:48:15.080521 2024] [socache_shmcb:debug] [pid 9980:tid 344] mod_socache_shmcb.c(469): AH00826: subcache_size = 15992
[Thu May 16 14:48:15.080521 2024] [socache_shmcb:debug] [pid 9980:tid 344] mod_socache_shmcb.c(471): AH00827: subcache_data_offset = 2128
[Thu May 16 14:48:15.080521 2024] [socache_shmcb:debug] [pid 9980:tid 344] mod_socache_shmcb.c(473): AH00828: subcache_data_size = 13864
[Thu May 16 14:48:15.080521 2024] [socache_shmcb:debug] [pid 9980:tid 344] mod_socache_shmcb.c(475): AH00829: index_num = 88
[Thu May 16 14:48:15.080521 2024] [socache_shmcb:info] [pid 9980:tid 344] AH00830: Shared memory socache initialised
[Thu May 16 14:48:15.080521 2024] [ssl:info] [pid 9980:tid 344] AH01887: Init: Initializing (virtual) servers for SSL
[Thu May 16 14:48:15.095526 2024] [ssl:info] [pid 9980:tid 344] AH01876: mod_ssl/2.4.58 compiled against Server: Apache/2.4.58, Library: OpenSSL/3.1.5
[Thu May 16 14:48:15.095526 2024] [http2:info] [pid 9980:tid 344] AH03090: mod_http2 (v2.0.22, feats=, nghttp2 1.59.0), initializing...
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9980:tid 344] mpm_winnt.c(1716): AH00453: Child process is running
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9980:tid 344] mpm_winnt.c(344): AH00391: Child: Retrieved our scoreboard from the parent.
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9980:tid 344] mpm_winnt.c(466): AH00403: Child: Waiting for data for listening socket 0.0.0.0:443
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9808:tid 352] mpm_winnt.c(513): AH00408: Parent: Duplicating socket 660 (0.0.0.0:443) and sending it to child process 9980
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9808:tid 352] mpm_winnt.c(513): AH00408: Parent: Duplicating socket 656 ([::]:443) and sending it to child process 9980
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9808:tid 352] mpm_winnt.c(513): AH00408: Parent: Duplicating socket 652 (0.0.0.0:80) and sending it to child process 9980
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9808:tid 352] mpm_winnt.c(513): AH00408: Parent: Duplicating socket 476 ([::]:80) and sending it to child process 9980
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9808:tid 352] mpm_winnt.c(532): AH00411: Parent: Sent 4 listeners to child 9980
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9980:tid 344] mpm_winnt.c(466): AH00403: Child: Waiting for data for listening socket [::]:443
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9980:tid 344] mpm_winnt.c(466): AH00403: Child: Waiting for data for listening socket 0.0.0.0:80
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9980:tid 344] mpm_winnt.c(466): AH00403: Child: Waiting for data for listening socket [::]:80
[Thu May 16 14:48:15.127246 2024] [mpm_winnt:debug] [pid 9980:tid 344] mpm_winnt.c(491): AH00407: Child: retrieved 4 listeners from parent
[Thu May 16 14:48:15.127246 2024] [http2:info] [pid 9980:tid 344] h2_workers: created with min=550 max=825 idle_ms=600000
[Thu May 16 14:48:15.134249 2024] [mpm_winnt:debug] [pid 9980:tid 344] child.c(986): AH00352: Child: Acquired the start mutex.
[Thu May 16 14:48:15.134249 2024] [mpm_winnt:notice] [pid 9980:tid 344] AH00354: Child: Starting 550 worker threads.
[Thu May 16 14:48:15.143252 2024] [mpm_winnt:debug] [pid 9980:tid 10136] child.c(430): AH00334: Child: Accept thread listening on 0.0.0.0:80 using AcceptFilter none
[Thu May 16 14:48:15.143252 2024] [mpm_winnt:debug] [pid 9980:tid 10844] child.c(430): AH00334: Child: Accept thread listening on 0.0.0.0:443 using AcceptFilter none
[Thu May 16 14:48:15.143252 2024] [mpm_winnt:debug] [pid 9980:tid 10848] child.c(430): AH00334: Child: Accept thread listening on [::]:443 using AcceptFilter none
[Thu May 16 14:48:15.143252 2024] [mpm_winnt:debug] [pid 9980:tid 10868] child.c(430): AH00334: Child: Accept thread listening on [::]:80 using AcceptFilter none
Back to top
chrisiwien



Joined: 14 May 2024
Posts: 17
Location: Austria, Vienna

PostPosted: Thu 16 May '24 21:28    Post subject: Reply with quote

I found a post in the Apache Lounge forum, which suggest to switch to FastCGI.

I have managed that, but php_opcache crashes now frequently.

I set in php.ini
Code:
opcache.cache_id=${OPCACHE_ID}
and set the varible OPCACHE_ID in every V-Host section with
Code:
SetEnv OPCACHE_ID "[individual string]"
.

As long, as i call only one Website (one VHost), there are no problems, if i try to call another VHost on the server, an Internal Error 500 occurs, the Event Viewer in Windows shows this:

Code:
Faulting application name: php-cgi.exe, version: 8.1.21.0, time stamp: 0x64a52e58
Faulting module name: php_opcache.dll, version: 8.1.21.0, time stamp: 0x64a5329a
Exception code: 0xc0000005
Fault offset: 0x000000000000e08d
Faulting process id: 0x27bc
Faulting application start time: 0x01daa7c5eacabdde
Faulting application path: ...\php-cgi.exe
Faulting module path: ...\php_opcache.dll
Report Id: 1eced51b-d643-4d94-94ad-84ab406a8d0a
Faulting package full name:
Faulting package-relative application ID:


There is some mess with OPCache. Any ideas what I can do? Guess I have to setup something different for each VHost.
Back to top
Jan-E



Joined: 09 Mar 2012
Posts: 1266
Location: Amsterdam, NL, EU

PostPosted: Fri 17 May '24 8:58    Post subject: Reply with quote

Try twitching your opcache settings. With PHP 8.1.28 TS x64 as a Apache module I am successfully running OPcache.
Previously I had occasional OPcache errors, but not anymore:
Code:
opcache.mmap_base=0x0000700000000000
opcache.cache_id="xp81.x64"
opcache.memory_consumption=256
opcache.interned_strings_buffer=64
opcache.max_accelerated_files=32531
opcache.enable_cli=0
opcache.validate_permission=1
opcache.use_cwd=1
opcache.revalidate_path=1
opcache.enable_file_override=1
opcache.validate_timestamp=1
opcache.revalidate_freq=60
opcache.file_cache_fallback=1
opcache.file_cache="c:\\temp\\php81.x64"
opcache.error_log="c:\\temp\\opcache.php81.x64.log"
opcache.jit=off

c:\temp\php81.x64 is a directory. Current stats:
Code:

Optimization            Enabled
SHM Cache               Enabled
File Cache              Enabled
JIT                     Disabled
Startup                 OK
Shared memory model     win32
Cache hits              9923360
Cache misses            6794
Used memory             152196456
Free memory             116208392
Wasted memory           30608
Interned Strings Used memory   24370520
Interned Strings Free memory   42738344
Cached scripts          5097
Cached keys             5097
Max keys                32531
OOM restarts            0
Hash keys restarts      0
Manual restarts         0

I disabled JIT because it had stability issues. Besides that, with a 99.93% cache hit rate I do not need JIT anyway.
This Apache is running a Drupal 10 site. With more Drupal 10 sites I would reuse the same OPcache ID xp81.x64 at mmap_base 0x0000700000000000.

Edit: this worked as well with PHP 8.1.21. The mmap_base was chosen out of the 4 valid values for Win64:
https://github.com/php/php-src/blob/PHP-8.1.28/ext/opcache/shared_alloc_win32.c#L214


Last edited by Jan-E on Fri 17 May '24 10:11; edited 5 times in total
Back to top
James Blond
Moderator


Joined: 19 Jan 2006
Posts: 7371
Location: Germany, Next to Hamburg

PostPosted: Fri 17 May '24 8:59    Post subject: Reply with quote

The 0xc0000005 is an access violation. It is a problem accessing the memory (RAM).
You may try PHP 8.1.28, and it might help or you change the opcache from RAM to files. For sure you can create a little RAM disk so it isn't slower.
Or you disable opcache.
Back to top
Stray78



Joined: 15 Apr 2024
Posts: 23
Location: USA

PostPosted: Fri 17 May '24 12:16    Post subject: Reply with quote

Looks like you're running Apache 2.4.58, I would update to 2.4.59.

First thing I noticed was your mod security status engine query failing. You can turn that off in your mod_security config.

('SecStatusEngine Off'), The status engine's purpose is to send limited information about the deployment (e.g. which ModSecurity version are you using) to a central server for statistical purposes.

Second I have no problem adjusting my threads.
<IfModule mpm_winnt_module>
ThreadLimit 15000
ThreadsPerChild 15000
MaxConnectionsPerChild 0
</IfModule>

[Thu May 16 09:23:03.527741 2024] [mpm_winnt:notice] [pid 70144:tid 344] AH00354: Child: Starting 15000 worker threads.

I have never ran FastCGI I run PHP as a module since 2005. I run Windows Server 2022 with Apache/2.4.59 (Win64) OpenSSL/3.3.0 PHP/8.3.7


And not sure what this is as I never see any ssl entries like this in my log. (shows different version of mod_ssl & libraries)

[Thu May 16 14:48:08.821789 2024] [ssl:info] [pid 9808:tid 352] AH01887: Init: Initializing (virtual) servers for SSL
[Thu May 16 14:48:08.834294 2024] [ssl:info] [pid 9808:tid 352] AH01876: mod_ssl/2.4.58 compiled against Server: Apache/2.4.58, Library: OpenSSL/3.1.5

Do you see anything in your Windows event viewer when Apache crashes/restarts?
Back to top
chrisiwien



Joined: 14 May 2024
Posts: 17
Location: Austria, Vienna

PostPosted: Fri 17 May '24 14:28    Post subject: Reply with quote

First, thanks to all of you, i really appreciate your help. Very Happy

From time to time php_opcache.dll is faulting further. Example message in the apache error log:
Code:

[quote][Fri May 17 14:15:17.543611 2024] [fcgid:info] [pid 11472:tid 728] mod_fcgid: process .../php-cgi.exe(7080) exit(communication error), return code -1073741819[/quote]


The opcache error log shows at the same time massive flooding with this messages:

Code:

Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow
Fri May 17 14:15:22 2024 (20900): Warning Interned string buffer overflow


Also the event viewer show again a faulting php_opcache.dll at the same time stamp.

I switched to FastCGI, but left the Thread-Safe version of PHP. But, it is recommended to use the Non-Thread-Safe version for Apache for Windows with FastCGI: https://www.php.net/manual/en/install.windows.manual.php. Switching to Non-Thread-Safe could be also a problem solver in my case.


@Jan-E: You are right about JIT. As soon i deactivated JIT, the websites were reachable fast and without massive problems with opache, but sometimes erratic errors occured anyway (means: Internal Server Error, Event viewer shows a faulting php_opcache.dll).

Maybe it isn't enough just disabling JIT. An other user has commented out also the JIT buffer setting. For now, my opache jit section looks this way:
Code:
;opcache.jit_buffer_size=100M
;opcache.jit=1255
opcache.jit=disable


I can't verify, if my effort with opcache.cache_id helped anything, the opcache log shows also with opcache.log_verbosity_level=4 nothing about that.

I will think about your opcache settings, thanks for sharing.


@James Blond: Exactly, this is an access violation. I found meanwhile a very interesting conversation on github: https://github.com/php/php-src/issues/7915, User cmb69 explains what's this all about (i have to admit, i don't understand everything yet):
"However, sharing OPcache instances for "unrelated" (i.e. non forked) processes is a quirk on Windows, and can't generally work, since these processes (especially if dynamically loaded extensions are involved) can have different memory layout (ASLR), but OPcache partly relies on a fixed memory layout, and the only mitigation on Windows is checking that a single central function (zend_execute_ex) is at the same address when reattaching."

And yes, upgrading PHP would be the next logic step, i also found some conversations, which suggests upgrading due opcache-related errors. It seems, some php versions have OPCache bugs.

Meanwhile I activated the file cache for opcache - which is recommended on Windows machines, by the way.

I consider deactivating OPCache only as the last option, php processing would be very slow then.


@Stray78: Yeah, i think i have to update the Apache server, perhaps the messages like "mod_ssl/2.4.58 compiled against Server" will disappear then. Regarding php as a module: i don't know why, but the page loading problems with several website were huge. Switching to FastCGI solves the problems immediately. Since "it works" now, my issue with ThreadsPerChild has no longer top priority for me.
Back to top
chrisiwien



Joined: 14 May 2024
Posts: 17
Location: Austria, Vienna

PostPosted: Fri 17 May '24 14:37    Post subject: Reply with quote

@Jan-E: i set in php.ini
Code:
opcache.interned_strings_buffer=64

as it is defined in your opcache settings. My value was 8MB, guess that was to low.

Will report if this helps.

Update at 14:40: the "interned string buffer overflows"-errors are gone, after the last Apache server restart.

Only one single warning appears (in a bulk of hundreds of messages):
Code:
Fri May 17 14:37:23 2024 (3920): Warning Not enough free shared space to allocate 27824 bytes (19144 bytes free)


An a new type of error raised: Rolling Eyes
Code:
Fri May 17 14:42:55 2024 (10528): Warning Internal error: wrong size calculation: ...\cp.php start=0x0000021cf8c04040, end=0x0000021cf8c05d28, real=0x0000021cf8c05948

But this error seems not to break my sites...
Back to top
Jan-E



Joined: 09 Mar 2012
Posts: 1266
Location: Amsterdam, NL, EU

PostPosted: Fri 17 May '24 16:17    Post subject: Reply with quote

Try updating your memory_limit to 1024M, double of what you have now.
Back to top
chrisiwien



Joined: 14 May 2024
Posts: 17
Location: Austria, Vienna

PostPosted: Fri 17 May '24 16:56    Post subject: Reply with quote

Ok, done, the memory_limit is now set to 1024M, thank you.

Jan-E, can you tell me on wich calculation/estimation this value is based:
Code:
opcache.max_accelerated_files=32531
Back to top
Jan-E



Joined: 09 Mar 2012
Posts: 1266
Location: Amsterdam, NL, EU

PostPosted: Fri 17 May '24 17:17    Post subject: Reply with quote

chrisiwien wrote:
Ok, done, the memory_limit is now set to 1024M, thank you.

Jan-E, can you tell me on wich calculation/estimation this value is based:
Code:
opcache.max_accelerated_files=32531

IIRC, this was some sort of threshold in the OPcache settings. It worked, so I have used it ever since.
Back to top
chrisiwien



Joined: 14 May 2024
Posts: 17
Location: Austria, Vienna

PostPosted: Fri 17 May '24 17:39    Post subject: Reply with quote

I have done a quick search, and found this:
User „gere“ explains it: https://stackoverflow.com/questions/47330690/how-does-opcache-max-accelerated-files-actually-work

So, you have to „simply“ count the files in the opcache log, that are stored during a request, the setting value should be greater than the counted file amount.

My sites are WordPress/WooCommerce websites, so i am curious, how many script calls are get listed in the opcache log…
Back to top
Jan-E



Joined: 09 Mar 2012
Posts: 1266
Location: Amsterdam, NL, EU

PostPosted: Fri 17 May '24 17:51    Post subject: Reply with quote

phpinfo() will tell you how many scripts are cached. My OPcache stats a few posts before this one were derived from that.

And this is for a Wordpress site on Linux. No WooCommerce, but quite a lot of other modules:
Code:
Cache hits        104840455
Cache misses      20690
Used memory       415532720
Free memory       120990680
Wasted memory     347512
Interned Strings Used memory   39831592
Interned Strings Free memory   10499592
Cached scripts    13997
Cached keys       13997
Max keys          32531

OPcache settings:
Code:
opcache.memory_consumption=512
opcache.interned_strings_buffer=64
opcache.max_accelerated_files=32531
opcache.enable_cli=1
opcache.validate_permission=1
opcache.use_cwd=1
opcache.revalidate_path=1
opcache.enable_file_override=1
opcache.validate_timestamps=1
opcache.revalidate_freq=60


Last edited by Jan-E on Fri 17 May '24 18:31; edited 6 times in total
Back to top
chrisiwien



Joined: 14 May 2024
Posts: 17
Location: Austria, Vienna

PostPosted: Fri 17 May '24 18:04    Post subject: Reply with quote

Ah ok, will try that.
Back to top
Jan-E



Joined: 09 Mar 2012
Posts: 1266
Location: Amsterdam, NL, EU

PostPosted: Fri 17 May '24 18:28    Post subject: Reply with quote

See edit in my previous post. 99.98% cache hit rate.
Back to top
chrisiwien



Joined: 14 May 2024
Posts: 17
Location: Austria, Vienna

PostPosted: Sat 18 May '24 1:09    Post subject: Reply with quote

Ok, i think i managed it. I've tried a bit further with PHP 8.1.21, but opening the WP-Admin of the websites resulted in a desaster. Many PHP errors, 500 Failures and so on.

I deactived OPCache, and everythink worked fine - but the performance dropped quite a lot.

So, I installed PHP 8.2.19 NTS and now everything runs with OPCache and without any problems, with the OPCache-Settings Jan-E recommended.

In the fcgid section of httpd.conf i added a new value:
Code:
FcgidSpawnScoreUpLimit 20


So far, all errors are gone none, also WP-Admin works like a charm.

Thanks to all for the help! Smile
Back to top
Stray78



Joined: 15 Apr 2024
Posts: 23
Location: USA

PostPosted: Sat 18 May '24 3:59    Post subject: Reply with quote

Glad to hear you got it working! Smile
Back to top
Jan-E



Joined: 09 Mar 2012
Posts: 1266
Location: Amsterdam, NL, EU

PostPosted: Sat 18 May '24 16:37    Post subject: Reply with quote

chrisiwien wrote:
Ok, i think i managed it. I've tried a bit further with PHP 8.1.21, but opening the WP-Admin of the websites resulted in a desaster. Many PHP errors, 500 Failures and so on.

WooCommerce seems to be really resource hungry. See
https://wordpress.org/support/topic/wc-admin-pages-slow-unresponsive-and-return-error-retrieving-settings/page/3/#post-17540187

Just curious: what are your OPcache stats now? How many cached scripts?
Back to top
chrisiwien



Joined: 14 May 2024
Posts: 17
Location: Austria, Vienna

PostPosted: Mon 20 May '24 6:05    Post subject: Reply with quote

Example:

Cache hits: 80234
Cache misses: 3

so, nearly 100%.
Back to top


Reply to topic   Topic: Apache runs unstable, keeps restarting in an erratic way View previous topic :: View next topic
Post new topic   Forum Index -> Apache