Topic: curl/wget/powershell blocked from downloading Apache Binary

[r15ch13]

Hi, I would like to ask why curl/wget and powershell get blocked through their UserAgent from downloading the Apache Binaries?
If the UserAgent contains one of the above-mentioned strings the server responds with a location header redirecting to http://localhost.
By changing the UserAgent header to something else bypasses this blockade.
I imagine that this kills some automated scripts.

Stumbled over this while I was investigating a problem about downloading Apache via Scoop (#2334), which uses "PowerShell/5.1" in its UserAgent.

Is this the desired behavior?

[admin]

Blocking is there already for years. Reason is bandwidth, special there were automated scripts that are downloading all the downloads every hour, what makes no sense. Also there are scoops seen lately, which doing it also every hour, and that for downloads which changes few times a year.

We keep the blocking. Yes, there is a workaround. When we see too much traffic, we block an other way or even on ip level.


Btw.
The powershell block is since this month, because the above mentioned scoop behavior.

[r15ch13]

Scoop is a command-line installer which allows users to install Apache and other tools (just like apt-get/brew).
It will only download a file when a user runs scoop install apache.
By blocking the "PowerShell"-UserAgent users of Scoop can no longer download and install Apache.
https://github.com/lukesampson/scoop/blob/6681471cccf21d5a149aa99d3009f0de4d6d5465/lib/core.ps1#L30-L32

Scoop has a service that runs every hour to scrape websites for new versions to provide fast updates.
This service downloads files only when the version changes.
If the website provides checksums, it will only extract these without downloading the actual file (which is the case for Apache).

Description file for Apache:
https://github.com/lukesampson/scoop/blob/master/bucket/apache.json

• checkver: information for scraping the version
  
• autoupdate.hash: information for scraping checksums
  

There might be users which use Scoop to install Apache on virtual machines or continuous integration platforms like AppVeyor without caching the binaries.


Would it be okay to remove "PowerShell" part from Scoops UserAgent so that Scoop is allowed?
Should the update service be throttled reduce traffic?

[admin]

Fine, to remove word PowerShell from Scoops UA. I shall keep an eye on the traffic. Throttling not necessary.

Why a service runs every hour to check for new versions ?
It is "normal" to check once a day or a week.

BTW.
I saw in apache.json -replace 'c:/Apache24', \"$root\"

Not sure it still works in the next Apache:

Then in the config files we have:

Define SRVROOT "c:/Apache24"
ServerRoot "${SRVROOT}"

See the Apache Lounge changes at www.apachelounge.com/viewtopic.php?t=8016

[r15ch13]

Sometimes programs get updates two or three times a day (hotfixes), so we just let the service run every hour to keep everything as fresh as possible. It's simpler this way because it doesn't need to keep track of which program to check.

The script that runs after installing Apache just replaces all occurrences of "c:/Apache24" with "C:/Users/<name>/scoop/apps/apache/current".

It will just change it to:
Define SRVROOT "C:/Users/<name>/scoop/apps/apache/current"