Topic: APACHE issue with RCA and ECC on systems

[Sraahvan]

we are observing issue with Apache + ECC and Apache + RSA which are listed below


httpd configuration:

<VirtualHost *:443>
Header edit Set-Cookie (.*) "$1;Secure"
Header edit Set-Cookie ^((?!(siemens_automation_language|breadcrumb_autorefresh|sinema_DLS)).*)$ $1;HttpOnly;Secure
ServerAdmin sinema@localhost.com
ServerName "localhost"
SSLEngine on
SSLProxyEngine on
SecRuleEngine On
SSLCertificateFile ../SSLCertificates/server/cert.pem
SSLCertificateKeyFile ../SSLCertificates/server/key.pem
SSLCACertificateFile ../SSLCertificates/sinecnmsca/cacert.pem



in our application - 1 master and 100 slaves sending heartbeat every minute to the master (100 HB/min at master)

Apache with ECC :

Recently we had requirement to move to ECC - Apache + ECC certificate is used (curves namely secp256k1 or secp256r1 or secp384r1) . it works only for sometime and later the UI does not respond at all - issue is UI is very slow and later it throws error that the "localhost took time to respond" or "proxy error". In Apache error.log we could find below errors -

[Sat Nov 12 21:28:04.926797 2022] [socache_dbm:error] [pid 4280:tid 1092] (28)No space left on device: AH00808: Cannot store socache object to DBM file `C:/Apache24'

[Sat Nov 12 21:30:44.779314 2022] [mpm_winnt:error] [pid 4280:tid 2436] AH00326: Server ran out of threads to serve requests. Consider raising the ThreadsPerChild setting

[Mon Nov 14 07:22:54.300520 2022] [socache_dbm:error] [pid 4280:tid 1048] (28)No space left on device: AH00808: Cannot store socache object to DBM file `C:Apache24'

[Tue Nov 08 23:30:15.169189 2022] [proxy_http:error] [pid 8952:tid 1152] [client ::1:57555] AH01097: pass request body failed to [::1]:49115 (localhost) from ::1 ()

[Tue Nov 15 11:04:47.294809 2022] [proxy_http:error] [pid 8952:tid 1076] (70014)End of file found: [client ::1:54153] AH01102: error reading status line from remote server localhost:49115

We ran process monitor and started it on httpd process. after a while we see many socket CLOSE_WAIT leaks at Apache's end. which was not recovering on its own . Restart of apache was the only solution here

this issue of apache with ECC is observed while using tlsv1.2 as well as tlsv1.3


Apache with RSA :

when used with tlsv1.2
Apache + RSA certificate (key length 2048/3072)= works fine. No issues were observed for over 5 years

but when used with tlsv1.3
all the above mentioned issues for apache with ECC were observed here too.

[James Blond]

I had the same issues with the curves. Current working config

[Sraahvan]

Thanks for the reply ,
config changes suggested not fixing the issue and We are still observing below mentioned issue with Apache + any End Entity certificate issued by ECC based CA"


httpd configuration:

<VirtualHost *:443>
Header edit Set-Cookie (.*) "$1;Secure"
Header edit Set-Cookie ^((?!(siemens_automation_language|breadcrumb_autorefresh|sinema_DLS)).*)$ $1;HttpOnly;Secure
ServerAdmin sinema@localhost.com
ServerName "localhost"
SSLEngine on
SSLProxyEngine on
SecRuleEngine On
SSLCertificateFile ../SSLCertificates/server/cert.pem
SSLCertificateKeyFile ../SSLCertificates/server/key.pem
SSLCACertificateFile ../SSLCertificates/sinecnmsca/cacert.pem

In our application we have 1 master and 100 slaves and slaves are sending heartbeat every minute to the master (100 HB/min received at master)
Apache is configured with End Entity certificate (RSA or ECC based End Entity) issued by ECC based CA. The End entity certificate is signed with sha256ECDSA signature algorithm.
Apache works fine if RSA or ECC based End Entity certfificate is signed by RSA based parent CA (sha256RSA signature algorithm).
But Apache becomes slow and unresponsive if the if RSA or ECC based End Entity certfificate is signed by ECC based parent CA (sha256ECDSA signature algorithm)

The issus is it works only for sometime and later the UI does not respond at all - issue is UI is very slow and later it throws error that the "localhost took time to respond" or "proxy error". In Apache error.log we could find below errors -

[socache_dbm:error] [pid 4280:tid 1092] (28)No space left on device: AH00808: Cannot store socache object to DBM file `C:/Apache24'
[mpm_winnt:error] [pid 4280:tid 2436] AH00326: Server ran out of threads to serve requests. Consider raising the ThreadsPerChild setting
[socache_dbm:error] [pid 4280:tid 1048] (28)No space left on device: AH00808: Cannot store socache object to DBM file `C:Apache24'
[proxy_http:error] [pid 8952:tid 1152] [client ::1:57555] AH01097: pass request body failed to [::1]:49115 (localhost) from ::1 ()
[proxy_http:error] [pid 8952:tid 1076] (70014)End of file found: [client ::1:54153] AH01102: error reading status line from remote server localhost:49115

We ran process monitor and started it on httpd process. after a while we see many socket CLOSE_WAIT leaks at Apache's end. which was not recovering on its own . Restart of apache was the only solution here

This issue of apache with End Entity certificate signed with sha256ECDSA signature algorithm is observed while using tlsv1.2 as well as tlsv1.3

[James Blond]

The first thing that I see is "No space left on device" aka your C: disk is full.

[Sraahvan]

"C: has enough space, over number of GBs". So low space in c: is not the issue. Kindly help

[James Blond]

Another idea is insufficient permissions.
What are your SSLSessionCache and SSLStaplingCache set to?

You may try

[Sraahvan]

Hi ,

Already these two cache settings are present and values are set to 512000

We are facing this issue in Apache only when the root CA or the signing CA is of ECC type. If the signing CA is of ECC type, and if the signature algorithm of the child Certificate is sha256ECDSA, then Apache becomes slow during handshake and lots of close_wait states observed. But when the signing CA is of RSA type (sha256RSA), Apache runs fine and we do not see this issue.