logo
Apache Lounge
Webmasters

 

About Forum Index Downloads Search Register Log in RSS X


Keep Server Online

If you find the Apache Lounge, the downloads and overall help useful, please express your satisfaction with a donation.

or

Bitcoin

A donation makes a contribution towards the costs, the time and effort that's going in this site and building.

Thank You! Steffen

Your donations will help to keep this site alive and well, and continuing building binaries. Apache Lounge is not sponsored.
Post new topic   Forum Index -> Third-party Modules View previous topic :: View next topic
Reply to topic   Topic: Available mod_qos :: updated Page 1, 2  Next
Author
Steffen
Moderator


Joined: 15 Oct 2005
Posts: 3092
Location: Hilversum, NL, EU

PostPosted: Tue 15 Mar '22 13:29    Post subject: Available mod_qos :: updated Reply with quote

03 June 2023 update to v11.74, see below

mod_qos is a quality of service (QoS) module for the Apache HTTP server implementing control mechanisms that can provide different priority to different requests.

For years only on *nix. And now our HTTPD developer Rainer Jung has made code changes to support Windows, many Thanks!

The module is able to protect your server from various kinds of malicious access or attacks. It has tons of options, including for DoS attacks, limit the number of requests to a URL, limit the number of concurrent connections etc.

Download : https://www.apachelounge.com/download/


Documentation http://mod-qos.sourceforge.net/ and https://en.wikipedia.org/wiki/Mod_qos


Discussion with Rainer can you find at https://www.apachelounge.com/viewtopic.php?t=8854
there is also a smoke test example.

Config :

Enable /mod_unique_id.so

LoadModule qos_module modules/mod_qos.so

<Location /qos>
SetHandler qos-viewer
</Location>

http://localhost/qos gives you an overview.


Enjoy

Note:
Maybe better for people needing to do load limiting it is a much better solution than starting with mod_security just for that purpose. And the other alternatives, like mod_evasive and mod_bw are rather limited.


Last edited by Steffen on Sat 03 Jun '23 16:37; edited 6 times in total
Back to top
puertoblack2003



Joined: 31 Jul 2009
Posts: 121
Location: U.S

PostPosted: Wed 16 Mar '22 15:50    Post subject: Reply with quote

testing now.So far no issue :thumbsup:
Back to top
puertoblack2003



Joined: 31 Jul 2009
Posts: 121
Location: U.S

PostPosted: Wed 16 Mar '22 16:13    Post subject: Reply with quote

so lookin at this in my log

Code:
[Wed Mar 16 10:04:57.947996 2022] [qos:error] [pid 21776:tid 31752] [client 134.122.112.12:42146] mod_qos(045): access denied, invalid request line: can't parse uri, c=134.122.112.12, id=XGsQZlbaBQAaVQAbdQcAAAh8AAA17TF7
[Wed Mar 16 10:04:58.535895 2022] [core:error] [pid 21776:tid 31752] [client 134.122.112.12:42510] AH10244: invalid URI path (/cgi-bin/.%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/hosts)


the module is working properly?
Back to top
rjung



Joined: 26 Aug 2015
Posts: 13

PostPosted: Thu 17 Mar '22 11:43    Post subject: Reply with quote

The core:error line is independent of mod_qos and would occur even without mod_qos loaded. So apache itself rightfully denies that request as a bad request. The %2e is percent encoding for ".", so someone tries to make a path traversal attack including many /../ but decodes them hoping this will circumvent for access rules. mod_qos has a check, whether Apache could successfully decode the URL (it could not in this case) and logs its own warning in addition to Apache. The overall web server behavior does not change in this case.

You can try with and without mod_qos e.g. using a http client, that does not itself decode the percent encoded URL but instead sends it as is to the server. e.g.:

curl -v -k "https://myserver/cgi-bin/.%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/hosts"

Thanks for testing and using the module!

Best regards,

Rainer
Back to top
Steffen
Moderator


Joined: 15 Oct 2005
Posts: 3092
Location: Hilversum, NL, EU

PostPosted: Thu 17 Mar '22 14:05    Post subject: Reply with quote

Did some testing:

When I set:
Code:
 QS_ResponseHeaderFilter on
Then Apache does not start, in event viewer:
Code:
 QS_ResponseHeaderFilter not allowed here


Had set:
Code:
QS_RequestHeaderFilter  on
See for explanation http://mod-qos.sourceforge.net/#filter cheking the headers against http://mod-qos.sourceforge.net/headerfilterrules.txt

But the error.log was filled with warnings
Code:
[Thu Mar 17 12:12:21.074466 2022] [qos:warn] [pid 7096:tid 3024] [client 87.18.161.216:0] mod_qos(042): drop request header: 'Sec-Fetch-Dest: document', (no rule available), c=127.0.0.1, id=YjMXlf-hjA4tVBd_QRz-rgAAAP8
[Thu Mar 17 12:12:21.074466 2022] [qos:warn] [pid 7096:tid 3024] [client 87.18.161.216:0] mod_qos(042): drop request header: 'Sec-Fetch-Mode: navigate', (no rule available), c=127.0.0.1, id=YjMXlf-hjA4tVBd_QRz-rgAAAP8
[Thu Mar 17 12:12:21.074466 2022] [qos:warn] [pid 7096:tid 3024] [client 87.18.161.216:0] mod_qos(042): drop request header: 'Sec-Fetch-Site: none', (no rule available), c=127.0.0.1, id=YjMXlf-hjA4tVBd_QRz-rgAAAP8
[Thu Mar 17 12:12:21.074466 2022] [qos:warn] [pid 7096:tid 3024] [client 87.18.161.216:0] mod_qos(042): drop request header: 'Sec-Fetch-User: ?1', (no rule available), c=127.0.0.1, id=YjMXlf-hjA4tVBd_QRz-rgAAAP8
Looks like most of the headers are added bij the broswer.

So set the loglevel to log not the warnings:
Code:
LogLevel qos:error

Then I was getting with my email client access denied
Code:
 [qos:error] [pid 9968:tid 2984] [client 2a10:3781:2e1:1:d97d:e339:cff0:4e52:0] mod_qos(043): access denied, request header: 'Content-Type: multipart/form-data; charset=UTF-8; boundary="surgeweb_123"', (pattern=^(["a-zA-Z0-9*/; =-]+){1}([ ]?,[ ]?(["a-zA-Z0-9*/; =-]+))*$, max. length=200), c=127.0.0.1, id=YjMYP2nnPOO8tl9-oz5VOgAAAPs, referer: https://www.land10.nl/surgeweb
The mail client has indeed long headers.
Setting to
Code:
QS_RequestHeaderFilter size
solved the issue.

Running now without issues sofar.
Back to top
rjung



Joined: 26 Aug 2015
Posts: 13

PostPosted: Thu 17 Mar '22 14:47    Post subject: Reply with quote

Hi Steffen,
the two directives QS_RequestHeaderFilter and QS_ResponseHeaderFilter are decorated dirrferently in the source code of the module. Wheres as request one can be used inside and outside of <Directory> and <Location>, the response one must be used inside of <Directory> or <Location>. So that one can't be used directly top level in the global server config or a virtual host config. Not sure why this is the case, but at least that part of the code explains the "not allowed here". I could ask the author whether it is intentional, but as a workaround you could use <Location />. Not saying it is actually a good general setting to activate the response header filter.

The warning when activating strict request header validation are probably expected. One would have to add more allowed headers with QS_RequestHeaderFilterRule. But I think using this module for too much of request checking one slowly gets into the realm of mod_security. IMHO the strenghts of mod_qos are by enforcing definable metric limits as max concurrency, request and event rates. This limits can be defined very flexible and use to prevent from overload situations and also to react on application specific things like brute force login attacks.

Thanks and regards,

Rainer
Back to top
puertoblack2003



Joined: 31 Jul 2009
Posts: 121
Location: U.S

PostPosted: Thu 17 Mar '22 15:13    Post subject: Reply with quote

should we continue here in this thread results of testing?

steffen pointed out a sample config , which iI'm currently using.

Code:
<IfModule qos_module>
# handle connections from up to 100000 different IPs
QS_ClientEntries 100000

# allow only 50 connections per IP
QS_SrvMaxConnPerIP 50

#limit maximum number of active TCP connections limited to 256
ThreadsPerChild 1920

# disables keep-alive when 180 (70%) TCP connections are occupied
QS_SrvMaxConnClose 180

# minimum request/response speed
# (deny slow clients blocking the server, keeping connections open without requesting anything
QS_SrvMinDataRate 150 1200
</IfModule>

the only diffence is from maxclient I'm using threadsperchild

this is what i got from log.

Code:

[Wed Mar 16 23:36:30.908857 2022] [qos:error] [pid 37828:tid 16388] mod_qos(034): access denied, QS_SrvMinDataRate rule (in:0): min=151, this connection=0, c=152.32.189.9
[Wed Mar 16 23:36:36.911367 2022] [qos:error] [pid 37828:tid 16388] mod_qos(034): access denied, QS_SrvMinDataRate rule (in:0): min=151, this connection=0, c=152.32.189.9
[Wed Mar 16 23:36:42.913810 2022] [qos:error] [pid 37828:tid 16388] mod_qos(034): access denied, QS_SrvMinDataRate rule (in:0): min=154, this connection=0, c=152.32.189.9
[Wed Mar 16 23:42:34.079653 2022] [qos:error] [pid 37828:tid 16388] mod_qos(034): access denied, QS_SrvMinDataRate rule (in:0): min=156, this connection=0, c=118.193.34.168
[Wed Mar 16 23:42:40.083258 2022] [qos:error] [pid 37828:tid 16388] mod_qos(034): access denied, QS_SrvMinDataRate rule (in:0): min=155, this connection=0, c=118.193.34.168
[Wed Mar 16 23:42:46.086183 2022] [qos:error] [pid 37828:tid 16388] mod_qos(034): access denied, QS_SrvMinDataRate rule (in:0): min=153, this connection=0, c=118.193.34.168


The error recorded is that a config issue, or is it it suppose to log that way by design?
Back to top
rjung



Joined: 26 Aug 2015
Posts: 13

PostPosted: Thu 17 Mar '22 18:48    Post subject: Reply with quote

A comment on the log lines you observe:

[Wed Mar 16 23:42:46.086183 2022] [qos:error] [pid 37828:tid 16388] mod_qos(034): access denied, QS_SrvMinDataRate rule (in:0): min=153, this connection=0, c=118.193.34.168

This means, that client connections, that did not send enough data, were forcibly closed by the module.

The config

QS_SrvMinDataRate 150 1200

requires a minimum data rate of 150 Bytes/second but increases that minimum requirement depending on your current concurrency up until 1200 bytes/second when the server is busy. The "min=153" in the log line tells us, that there the module increased the required minimum from the 150 configures bytes/secod slightly to 153 bytes/second. connection=0 tells us, that these connections did not send anything at all. The checks happen every 5 seconds by default, but this granularity can be changed eg. to 10 or 20 seconds.

It seems to be working fine, but every now and then there is a connection hitting your server where the client does not immediately send a request or stalls during sending it. As your configuration denies such behavior, the module now kills those connections.

Best regards,

Rainer
Back to top
puertoblack2003



Joined: 31 Jul 2009
Posts: 121
Location: U.S

PostPosted: Fri 18 Mar '22 16:03    Post subject: Reply with quote

rjung wrote:
QS_SrvMinDataRate 150 1200

requires a minimum data rate of 150 Bytes/second but increases that minimum requirement depending on your current concurrency up until 1200 bytes/second when the server is busy. The "min=153" in the log line tells us, that there the module increased the required minimum from the 150 configures bytes/secod slightly to 153 bytes/second. connection=0 tells us, that these connections did not send anything at all. The checks happen every 5 seconds by default, but this granularity can be changed eg. to 10 or 20 seconds.

It seems to be working fine, but every now and then there is a connection hitting your server where the client does not immediately send a request or stalls during sending it. As your configuration denies such behavior, the module now kills those connections.


thanks for the heads up! so change the settings QS_SrvMinDataRate 150 1200 to let say QS_SrvMinDataRate 250 1200 keeping it under 1200, Is that your suggestion?
Back to top
dmye



Joined: 23 Nov 2021
Posts: 7

PostPosted: Tue 22 Mar '22 11:57    Post subject: Reply with quote

If using mod_qos_user_id suggest Cancel
Code:
mod_unique_id
This feature currently isn't supported on Windows NT
Then the prompt log
Code:
mod_unique_id not available (mod_qos generates simple request id if required)
But correct
Back to top
dmye



Joined: 23 Nov 2021
Posts: 7

PostPosted: Mon 04 Jul '22 4:47    Post subject: Reply with quote

QS_UserTrackingCookieName Resulting in multiple consecutive slash ('/')
Back to top
Steffen
Moderator


Joined: 15 Oct 2005
Posts: 3092
Location: Hilversum, NL, EU

PostPosted: Tue 08 Nov '22 13:09    Post subject: Reply with quote

Version 11.72 available only for VS17
Back to top
DnvrSysEngr



Joined: 15 Apr 2012
Posts: 226
Location: Denver, CO USA

PostPosted: Fri 18 Nov '22 5:18    Post subject: Reply with quote

After installing and configuring mod_qos, I am not getting this in my error log when launching apache:

loaded MPM is 'WinNT' but mod_qos should be used with MPM 'Worker' or 'Event' only.

What should be done to resolve this?
Back to top
admin
Site Admin


Joined: 15 Oct 2005
Posts: 692

PostPosted: Fri 18 Nov '22 8:15    Post subject: Reply with quote

You can ignore it for windows.
Back to top
Steffen
Moderator


Joined: 15 Oct 2005
Posts: 3092
Location: Hilversum, NL, EU

PostPosted: Fri 13 Jan '23 12:21    Post subject: Reply with quote

Updated to version 11.73. Only VS17

Most noticeable change it that it is now build with PCRE2.
Back to top
puertoblack2003



Joined: 31 Jul 2009
Posts: 121
Location: U.S

PostPosted: Sat 03 Jun '23 12:25    Post subject: Reply with quote

latest 11.74 https://sourceforge.net/projects/mod-qos/files/
Back to top
Steffen
Moderator


Joined: 15 Oct 2005
Posts: 3092
Location: Hilversum, NL, EU

PostPosted: Sat 03 Jun '23 16:38    Post subject: Reply with quote

Thanks !

Updated.

Changes 11.74

- Fixed: Potential counter overflow for early event detection
(increment before block) or log only mode.
Back to top
Stray78



Joined: 15 Apr 2024
Posts: 23
Location: USA

PostPosted: Fri 10 May '24 2:46    Post subject: Reply with quote

Any idea when 11.75 will be compiled? Thank you!
Back to top
Steffen
Moderator


Joined: 15 Oct 2005
Posts: 3092
Location: Hilversum, NL, EU

PostPosted: Fri 10 May '24 8:56    Post subject: Reply with quote

Change is minor, only :

- QS_ClientIpFromHeader supports other modules (e.g. mod_remoteip) setting a client address to the request record when using the special header name #USERAGENT_IP.

I put it on the to-do list, expect coming weeks.
Back to top
Stray78



Joined: 15 Apr 2024
Posts: 23
Location: USA

PostPosted: Tue 14 May '24 3:14    Post subject: Reply with quote

Thank you!
Back to top


Reply to topic   Topic: Available mod_qos :: updated View previous topic :: View next topic
Post new topic   Forum Index -> Third-party Modules Page 1, 2  Next