| Author |
|
maskego
Joined: 16 Apr 2010
Posts: 238
|
 Posted: Thu 13 Oct '11 11:56 Post subject: How to stop HTTrack mass catch site? |
 |
|
Is there any good module can stop machine tools like HTTrack,telport...etc?Catch machines make site has heavy loading.It's so disgusting.
I use mod_rewrite,but can't stop that.It's because machine catch tools can rename the user-agent name.
 |
|
| Back to top |
|
James Blond
Moderator
Joined: 19 Jan 2006
Posts: 7371
Location: Germany, Next to Hamburg
|
|
| Back to top |
|
maskego
Joined: 16 Apr 2010
Posts: 238
|
| Posted: Fri 14 Oct '11 2:03 Post subject: |
|
|
james:
But,that direction site shows note: | Code: | Warning:
Access control by User-Agent is an unreliable technique, since the User-Agent header can be set to anything at all, at the whim of the end user.
|
If that,how can I stop all catch machine tools to site?Is there any good idea to stop it?  |
|
| Back to top |
|
James Blond
Moderator
Joined: 19 Jan 2006
Posts: 7371
Location: Germany, Next to Hamburg
|
| Posted: Fri 14 Oct '11 11:12 Post subject: |
|
|
| Ya, it is not fully reliable. But it keeps some traffic out. Much better is to search the log files for it, collect the IPs and ban those IPs in your firewall that they don't reach apache at all. |
|
| Back to top |
|
maskego
Joined: 16 Apr 2010
Posts: 238
|
| Posted: Fri 14 Oct '11 11:19 Post subject: |
|
|
Sure,it's not reliable at all.
Is it possible to use some modules of apache to prevent web catch behavior from normal visit sites?(use modules to ban their behavior rather than ban their user-agent)
| James Blond wrote: | | Ya, it is not fully reliable. But it keeps some traffic out. Much better is to search the log files for it, collect the IPs and ban those IPs in your firewall that they don't reach apache at all. |
|
|
| Back to top |
|
James Blond
Moderator
Joined: 19 Jan 2006
Posts: 7371
Location: Germany, Next to Hamburg
|
| Posted: Fri 14 Oct '11 11:34 Post subject: |
|
|
| There is might a combinaton of Mod Limit IP Connection and Mod Bandwidth. At least Mod Limit IP Connection (can be found on apachehaus.com) can be very useful in your situation. |
|
| Back to top |
|
maskego
Joined: 16 Apr 2010
Posts: 238
|
| Posted: Sat 15 Oct '11 6:35 Post subject: |
|
|
Does mod_security can stop HTTrack webcopier tools behavior not user-agent?
And,how to set it? |
|
| Back to top |
|
