Topic: Apache reverse proxy Dotnet Upstream error

[tonysar]

Hello.
I am working on a site developed in Dotnet hosted on Linux Ubuntu vps , Apache 2 .
Apache is reverse proxy to dotnet core . Using Let's Encrypt SSL, Everything is working well with One single issue.

At times I have to start upstream server , being hosted on linux , init service script located on /etc/systemd/system will take about 10 seconds to restart upstream server .
because of this delay .. I get error 503
Proxy_http_error ; HTTP: failed to make connection to backend: 127.0.0.1.
back stream server assigned to localhost and listens on port 5000. this error due to stoped server.

I like to know , if there is way to fix this ? maybe somehow tell apache to wait for 10 seconds .
I found nothing in past 10 days on how to fix this little problem

Thanks.

[tangent]

You haven't posted the relevant section of your reverse proxy configuration, but assuming you're using the ProxyPass directive, along with ProxyTimeout, what parameter options have you configured?

ProxyTimeout defaults to the Apache Timeout setting, which is 60 seconds by default, so I'm surprised your back end server restarts are causing a problem, unless this figure has been changed.

Specifically for ProxyPass, have you set timeout, connectiontimeout and possibly retry? I'd also consider setting keepalive to on.

See https://httpd.apache.org/docs/current/mod/mod_proxy.html#proxypass and https://httpd.apache.org/docs/current/mod/mod_proxy.html#proxytimeout for details.

[tonysar]

Thanks.
here is the config of vhost .

[tangent]

In your configuration, the retry parameter is set to 1 second (the default is 60 seconds). If your backend service restart takes 10 seconds, then I'd set retry to be slightly greater than that time. This is what the mod_proxy_http docs say about the retry parameter.

retry 60 Connection pool worker retry timeout in seconds. If the connection pool worker to the backend server is in the error state, Apache httpd will not forward any requests to that server until the timeout expires. This enables to shut down the backend server for maintenance and bring it back online later. A value of 0 means always retry workers in an error state with no timeout.

You've also set the proxy (socket) timeout to 600 seconds, which means you're prepared to wait 10 mins for proxied content response from the backend. This may be perfectly reasonable, depending on what processing the backend is actually doing, e.g. report generation, but unless needed, it will cause Apache to hold on to resources that could otherwise be released sooner.

That said, your configuration looks somewhat confusing over the VirtualHost sections. Apart from the proxy entries, your port 80 section contains rewrite rules to redirect the client to an https connection on port 443. I'd therefore expect to see the proxy entries, and bulk of the configuration settings, in a port 443 virtualhost section.

If the above parameter changes don't help solve the problem, then up the loglevel for mod_proxy to see what's going on, e.g.

[tonysar]

Thank you so much for your help
I increased the retry time , didnt't work . as you have suggested, changed loglevel to show proxy . nothing really there to show reason behind this. except connection refused during backend boot .

As for vhost.
I used certbot to generate ssl , Certbot created second vhost for 443 post that is identical to what I have posted here .

other then that one issue everything seems to be working and there are no errors in log that would be of any concerns . now the question is why I can not make apache / Proxt wait for 10 seconds, is just Odd .

[tangent]

Ok, so it would seem the retry setting applies to existing connections, rather than creating new ones.

Even if the core Apache timeout hasn't been changed elsewhere in your configuration, I'd still explicitly set the ProxyTimeout, to localise the setting for your proxy service.

I'd also add connectiontimeout and ping parameters to your ProxyPass directive. If you read the documentation over ping, it causes Apache to test the connection first (waiting for the specified time), so this should help get over the backend restart problem, albeit for a slight increase in service overhead.

Adjust the parameter times to suit your system setup, e.g.