Topic: apache + mod_pagespeed + mod_http2 crash

[mirkofp]

Hi,

I manage a webserver where a quite high traffic ecommerce runs.

Last week 2 times Apache crashed during the night, and we are investigating the causes.

The log lines that alarm me are:

Nov 21 04:55:46 server-web kernel: [21138757.911742] cgroup: fork rejected by pids controller in /system.slice/apache2.service
Nov 21 07:19:35 server-web kernel: [21147387.212985] apache2[8666]: segfault at 57c ip 00007fc60f236f45 sp 00007fc5f5592cd0 error 4
Nov 21 07:19:35 server-web kernel: [21147387.212987] apache2[8668]: segfault at 57c ip 00007fc60f236f45 sp 00007fc5f7596cd0 error 4
Nov 21 07:19:35 server-web kernel: [21147387.212990] in mod_pagespeed_ap24.so[7fc60f18b000+9a0000]

and then 2 days later:

Nov 23 01:32:51 eurobrico-web kernel: [21299380.928686] apache2[1643]: segfault at 7f7c3ded7708 ip 00007f7c3e474d0c sp 00007f7c26cf7e40 error 4 in mod_http2.so[7f7c3e467000+24000]
Nov 23 01:32:51 eurobrico-web kernel: [21299380.928694] Code: 85 ff 74 05 e8 35 2e ff ff 48 8b bb 90 00 00 00 48 83 c4 08 5b 5d e9 b3 25 ff ff 0f 1f 00 48 8b bf 90 00 00 00 e8 a4 25 ff ff <48> 8b 7b 50 8b 75 00 e8 48 3f 01 00 85 c0 74 14 48 8b bb 90 00 00
Nov 23 01:49:56 eurobrico-web kernel: [21300405.509228] cgroup: fork rejected by pids controller in /system.slice/apache2.service

disabling mod_pagespeed seems to have made situation stable, but we'd like to understand exactly what happened.

I suppose it has something to do with pagespeed and http2 not working well together.

It wasn't a traffic spike or some attack, traffic was regular. In the night some scheduled task run but nothing too much intensive. From VM RAM memory graphs I see an increase to 95% of RAM used in that moment.

Server is running Apache/2.4.38 (Debian) with mpm_event
with those modules:
core_module (static)
so_module (static)
watchdog_module (static)
http_module (static)
log_config_module (static)
logio_module (static)
version_module (static)
unixd_module (static)
access_compat_module (shared)
alias_module (shared)
auth_basic_module (shared)
authn_core_module (shared)
authn_file_module (shared)
authz_core_module (shared)
authz_host_module (shared)
authz_user_module (shared)
autoindex_module (shared)
deflate_module (shared)
dir_module (shared)
env_module (shared)
expires_module (shared)
fcgid_module (shared)
filter_module (shared)
headers_module (shared)
http2_module (shared)
mime_module (shared)
mpm_event_module (shared)
negotiation_module (shared)
proxy_module (shared)
proxy_fcgi_module (shared)
reqtimeout_module (shared)
rewrite_module (shared)
setenvif_module (shared)
socache_shmcb_module (shared)
ssl_module (shared)
status_module (shared)

PHP 7.3.31-1~deb10u4
(with PHP FPM)


If someone had some similar issues, let me know. Many thanks!

[James Blond]

What is in your apache error log about this?

[mirkofp]

Hi! thanks for the answer.

I paste here:
https://apaste.info/9xp5

the error.log of Apache for the first crash, the whole night log until our manual Apache restart at 07:19 AM

[csdude55]

I can't speak to the exact nature of your errors, but I can say that I've recently been fielding tons of errors related to mod_pagespeed, too! I disabled it, and they all went away.

It could be a coincidence, but I started using Cloudflare about 3 months ago. I didn't see a significant speed increase after implementing Cloudflare, but I didn't see a decrease after disabling pagespeed, either.

Based on this, you might consider using Cloudflare as an alternative.

[mirkofp]

hi! thanks for the answer!
Yes, i see other user suggesting that Pagespeed is an old project, to use something else.

Thanks.