Author |
|
lavesa
Joined: 06 Jan 2006 Posts: 13
|
Posted: Wed 01 Mar '06 2:38 Post subject: Event Viewer errors and logged errors |
|
|
Hi,
PLEASE HELP!!!
I've recently installed 2.2 on Windows 2003 Server as a reverse proxy. Periodically, I get a stream of Information events and then Error events with the following. At one point, it actually disabled the server.
Faulting application httpd.exe, version 2.2.0.0, faulting module libaprutil-1.dll, version 1.2.2.0, fault address 0x00002b17.
Faulting application httpd.exe, version 2.2.0.0, faulting module mod_proxy.so, version 0.0.0.0, fault address 0x000061ad.
When I check the error log, there are corresponding messages:
[Tue Jan 24 18:13:54 2006] [error] [client 10.0.22.158] proxy: error reading status line from remote server rchportwt.na.lennoxintl.com, referer: http://rchportwt.na.lennoxintl.com/portal/portal_redirect.asp
[Tue Jan 24 18:13:54 2006] [error] [client 10.0.22.158] proxy: Error reading from remote server returned by /irj/portal/adp, referer: http://rchportwt.na.lennoxintl.com/portal/portal_redirect.asp
[Tue Jan 24 18:26:54 2006] [error] (OS 10061)No connection could be made because the target machine actively refused it. : proxy: HTTP: attempt to connect to 192.168.150.160:50000 (rchportwt.na.lennoxintl.com) failed
[Tue Jan 24 18:26:54 2006] [error] ap_proxy_connect_backend disabling worker for (rchportwt.na.lennoxintl.com)
[Tue Jan 24 18:26:54 2006] [error] proxy: HTTP: disabled connection for (rchportwt.na.lennoxintl.com)
Can someone shed some light on this?? I have this on a production server and I need to make sure I have everything configured properly. Here is my config file which is using Virtual Hosts:
Admin Note:
httpd.conf removed, see forum rules |
|
Back to top |
|
lavesa
Joined: 06 Jan 2006 Posts: 13
|
Posted: Wed 01 Mar '06 2:46 Post subject: |
|
|
Additional Info:
Most of these errors do not cause the server to shut down. An example is
Event Type: Error
Event Source: Application Error
Event Category: (100)
Event ID: 1000
Date: 2/20/2006
Time: 12:24:53 PM
User: N/A
Computer: RCHPORTWT
Description:
Faulting application httpd.exe, version 2.2.0.0, faulting module mod_proxy.so, version 0.0.0.0, fault address 0x000061ad.
Is there a limit on the size of the URL?
Thanks again,
Andrew |
|
Back to top |
|
James Blond Moderator
Joined: 19 Jan 2006 Posts: 7373 Location: Germany, Next to Hamburg
|
Posted: Wed 01 Mar '06 13:09 Post subject: |
|
|
Seems to be a bad libaprutil-1.dll. Where did you downloaded it? From this page?
Also have a look at http://support.microsoft.com/kb/261007/en-us
That is might a DNS error give the server a name not a IP ServerName 10.0.2.195:80 Name it to localhost:80
Also change <Directory "E:/Program Files/Apache Group/Apache2/htdocs"> to <Directory "e:/wwwroot/lennox"> |
|
Back to top |
|
lavesa
Joined: 06 Jan 2006 Posts: 13
|
Posted: Wed 01 Mar '06 18:53 Post subject: |
|
|
Hi,
Thanks for the reply. Yes, I did download Apache from this site. I also confirmed that the DNS server is properly configured on the server.
I'm not as concerned about the server being disabled now because I realized the two times this happened it was because Apache could not reach the destination server. I've added my load balancer to the local host file to hopefully avoid this in the future.
I just need to clean up all the errors in the Event viewer. I also get a stream of warnings (1004). When an event happens, it appears to be in groups of about 10. Eight are caused by mod_proxy.so and two are because of libaprutil-1.dll. Is this significant? Do I have corrupted files? The odd thing is that I can go days with no errors and then get a stream of them.
Event Type: Information
Event Source: Application Error
Event Category: (100)
Event ID: 1004
Date: 2/27/2006
Time: 11:02:16 PM
User: N/A
Computer: RCHPORTWT
Description:
Reporting queued error: faulting application httpd.exe, version 2.2.0.0, faulting module mod_proxy.so, version 0.0.0.0, fault address 0x000061ad.
I've made all the changes you've suggested so I'll see if this makes a difference.
Thanks,
Andrew |
|
Back to top |
|
Steffen Moderator
Joined: 15 Oct 2005 Posts: 3092 Location: Hilversum, NL, EU
|
Posted: Thu 02 Mar '06 0:08 Post subject: |
|
|
There is an issue with mod_proxy using Reverse proxy. in the area of the keep-alive.
The following is from the change log in the dev-tree:
*) mod_proxy_http: Do send keep-alive header if the client sent
connection: keep-alive and do not close backend connection if the client
sent connection: close. PR 38524. [Ruediger Pluem, Joe Orton]
*) mod_proxy: Fix KeepAlives not being allowed and set to
backend servers. PR 38602. [Ruediger Pluem, Jim Jagielski]
*) mod_proxy: If we get an error reading the upstream response,
close the connection.
[Justin Erenkrantz, Roy T. Fielding, Jim Jagielski, Ruediger Pluem]
As soon as the official code is available, I shall build.
I am also running Reverse Proxy, but I am lucky, not seen your errors.
Steffen |
|
Back to top |
|
lavesa
Joined: 06 Jan 2006 Posts: 13
|
Posted: Thu 02 Mar '06 1:04 Post subject: |
|
|
Hi Steffen,
So this is a known bug? How will I know when the updates will be available?
Thanks,
Andrew |
|
Back to top |
|
riviereg
Joined: 20 Jan 2006 Posts: 6
|
Posted: Sat 18 Mar '06 6:54 Post subject: mod_proxy crash my apache2.2 |
|
|
Dear all,
We get exactly the same problem, trying to proxy
an IIS on the same computer. This crash apache 2.2.
The windows evens show us the same even than Andrew.
Is there a way to correct this ?
(I use Apache 2.2 from here on a windows 2003 server)
Guillaume |
|
Back to top |
|
James Blond Moderator
Joined: 19 Jan 2006 Posts: 7373 Location: Germany, Next to Hamburg
|
Posted: Sun 19 Mar '06 13:33 Post subject: |
|
|
I testet that out. This is a Windows 2003 error. On other Windows versions apache runs well. Under small buissness server 2003 this error also appear. I couldn't figure out what the error causes. |
|
Back to top |
|
riviereg
Joined: 20 Jan 2006 Posts: 6
|
Posted: Mon 20 Mar '06 4:15 Post subject: Trying to solve this problem |
|
|
I'd like to ask for opinion on this problem.
I'd need to solve this problem, what is the best way ?
1/ Re-install apache 2.0.55 with open ssl ...
2/ re-compile mod_proxy with windows 2003
3/ re-compile all apache on windows 2003
Thank for advices
Guillaume |
|
Back to top |
|
James Blond Moderator
Joined: 19 Jan 2006 Posts: 7373 Location: Germany, Next to Hamburg
|
Posted: Mon 20 Mar '06 9:42 Post subject: |
|
|
I think the fastest way will be re-install 2.0.x
Secondly try to compile Apache 2.2.x unter Win2k3. If you get run it on after compiling send us a copy. I didn't get it run under w2k3.
-----------------------
edit
I tried out the Apache 2.2.0 from this page with PHP 5 as normal proxy.
On Windows server 2003 Build 3790 with SP 1 and all updates
This config will run on every Driveletter also USB Stick
Folderstructure example on Drive C:
Code: |
C:\server
C:\server\apache\*
C:\server\logs
C:\server\www
|
For PHP support remove the comment # |
|
Back to top |
|
riviereg
Joined: 20 Jan 2006 Posts: 6
|
Posted: Tue 21 Mar '06 6:54 Post subject: Solved ? |
|
|
Dear James,
I may have found a solution for this problem, but i'm not sure because i don't know how this problem occur, so i can not reproduce it, just wait ...
I fact I check my evens and I see other errors from Windows 2003, specialy this one:
An I/O operation initiated by the Registry failed unrecoverably. The Registry could not read in or write out or flush one of the files that contain the system's image of the Registry.
I search for this problem and I did find a pach for windows 2003 servers:
http://www.eventid.net/display.asp?eventid=333&eventno=5757&source=Application%20Popup&phase=1
I apply the latest patch provided by microsoft on this tcpip.sys:
http://support.microsoft.com/?kbid=898060
And it may have solve my problem. I'm not sure yet (only try during one day) I will update this thread at the end of the week if all is ok.
But for now, no more strange evens and no more Apache 2.2 crash.
Thanks for your help,
Guillaume |
|
Back to top |
|
James Blond Moderator
Joined: 19 Jan 2006 Posts: 7373 Location: Germany, Next to Hamburg
|
Posted: Tue 21 Mar '06 10:41 Post subject: |
|
|
Yepp, was an windows 2003 issue also some x64 Versions. The hotfixes from http://support.microsoft.com/?kbid=898060 have that solution. Interessting is, that a hot from SP1 caused this error. With the german version of Windows 2003 this error didn't occurred. But when I installed the english Version on VMware I had the same error. Strange! |
|
Back to top |
|
riviereg
Joined: 20 Jan 2006 Posts: 6
|
Posted: Thu 23 Mar '06 3:52 Post subject: Bad news ... |
|
|
Well ... this not solve our problem. But this is a
well know bug of apache (#37770)
http://issues.apache.org/bugzilla/show_bug.cgi?id=37770
I'd like to keep my apache 2.2 on this windows 2003,
so I will try the re-compilation ...
This bug occurs just after some problems to contact the remote server.
I beleived that a problem occurs on my remote server (IIS 6.0 on port 9090;, on the same server) and this "break" in some way mod_proxy and after it raise a lot of those error.
this is my stack trace in evens viewer and error.log:
The first error in the windows evens is at 1:07:43 PM:
Faulting application httpd.exe, version 2.2.0.0, faulting module libaprutil-1.dll, version 1.2.2.0, fault address 0x00002b17.
1st Error in error.log:
[Wed Mar 22 12:51:39 2006] [error] [client x.x.x.X] (20023)The given path was above the root path: Cannot map GET / HTTP/1.0 to file
[Wed Mar 22 12:51:39 2006] [error] [client x.x.x.x] (20023)The given path was above the root path: Cannot map GET / HTTP/1.0 to file
[Wed Mar 22 13:07:45 2006] [notice] Parent: child process exited with status 3221225477 -- Restarting.
[Wed Mar 22 13:07:45 2006] [warn] RSA server certificate CommonName (CN) `XXX' does NOT match server name!?
[Wed Mar 22 13:07:45 2006] [notice] Apache/2.2.0 (Win32) mod_ssl/2.2.0 OpenSSL/0.9.8a configured -- resuming normal operations
[Wed Mar 22 13:07:45 2006] [notice] Server built: Dec 30 2005 21:24:41
[Wed Mar 22 13:07:45 2006] [notice] Parent: Created child process 2428
[Wed Mar 22 13:07:46 2006] [warn] Init: Session Cache is not configured [hint: SSLSessionCache]
[Wed Mar 22 13:07:46 2006] [warn] RSA server certificate CommonName (CN) `XXX' does NOT match server name!?
[Wed Mar 22 13:07:46 2006] [warn] RSA server certificate CommonName (CN) `XXX' does NOT match server name!?
[Wed Mar 22 13:07:46 2006] [notice] Child 2428: Child process is running
[Wed Mar 22 13:07:46 2006] [notice] Child 2428: Acquired the start mutex.
[Wed Mar 22 13:07:46 2006] [notice] Child 2428: Starting 250 worker threads.
[Wed Mar 22 13:07:46 2006] [notice] Child 2428: Starting thread to listen on port 443.
[Wed Mar 22 13:07:46 2006] [notice] Child 2428: Starting thread to listen on port 80.
[Wed Mar 22 13:08:01 2006] [error] [client x.x.x.x] proxy: error reading status line from remote server (null), referer: http://www.xxx.com/xxx/xxx.html
[Wed Mar 22 13:08:01 2006] [error] [client x.x.x.x] proxy: Error reading from remote server returned by /xxx/xxx/xxx.gif, referer: http://www.xxx.com/xxx/xxx.html
[Wed Mar 22 13:08:13 2006] [notice] Parent: child process exited with status 3221225477 -- Restarting.
After:
[Wed Mar 22 14:14:05 2006] [error] [client x.x.x.x] proxy: error reading status line from remote server (null), referer: http://xxx.com/
[Wed Mar 22 14:14:05 2006] [error] [client x.x.x].x proxy: Error reading from remote server returned by /xx/xx/xx
etc ... |
|
Back to top |
|
lavesa
Joined: 06 Jan 2006 Posts: 13
|
Posted: Tue 09 May '06 23:46 Post subject: What's the final verdict |
|
|
Hi All,
What is the final word on a solution here? Is this a Windows 2003 bug or a known issue with Apache? Also, is this a conflict with IIS? I only have an FTP server running and the web server is stopped.
Thanks,
Andrew |
|
Back to top |
|
lavesa
Joined: 06 Jan 2006 Posts: 13
|
Posted: Tue 09 May '06 23:54 Post subject: |
|
|
James Blond wrote: | Yepp, was an windows 2003 issue also some x64 Versions. The hotfixes from http://support.microsoft.com/?kbid=898060 have that solution. Interessting is, that a hot from SP1 caused this error. With the german version of Windows 2003 this error didn't occurred. But when I installed the english Version on VMware I had the same error. Strange! |
Are you saying this KB article solved the problem for running Apache on 2003 server? What was the process because the resolution section of this article is fairly confusing!
Thanks,
Andrew |
|
Back to top |
|
lavesa
Joined: 06 Jan 2006 Posts: 13
|
Posted: Tue 16 May '06 22:04 Post subject: Microsoft patches solved our problem |
|
|
Hi,
Just to close the loop on this... Once I applied all the Microsoft hotfixes for the 2003 server, it appears the problems have stopped in regards to Apache.
Thanks for all the help.
Andrew |
|
Back to top |
|
lavesa
Joined: 06 Jan 2006 Posts: 13
|
Posted: Fri 25 Aug '06 18:03 Post subject: ap_proxy_connect_backend disabling worker for |
|
|
Steffen wrote: | There is an issue with mod_proxy using Reverse proxy. in the area of the keep-alive.
The following is from the change log in the dev-tree:
*) mod_proxy_http: Do send keep-alive header if the client sent
connection: keep-alive and do not close backend connection if the client
sent connection: close. PR 38524. [Ruediger Pluem, Joe Orton]
*) mod_proxy: Fix KeepAlives not being allowed and set to
backend servers. PR 38602. [Ruediger Pluem, Jim Jagielski]
*) mod_proxy: If we get an error reading the upstream response,
close the connection.
[Justin Erenkrantz, Roy T. Fielding, Jim Jagielski, Ruediger Pluem]
As soon as the official code is available, I shall build.
I am also running Reverse Proxy, but I am lucky, not seen your errors.
Steffen |
Hi Steffen,
Do you know if a fix is out yet? I'm getting issues in production on a regular basis now with the worker process being disabled.
Code: |
[Wed Apr 05 15:15:53 2006] [error] (OS 10060)A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. : proxy: HTTP: attempt to connect to 10.0.2.223:50000 (rchportplb.na.lennoxintl.com) failed
[Wed Apr 05 15:15:53 2006] [error] ap_proxy_connect_backend disabling worker for (rchportplb.na.lennoxintl.com)
[Wed Apr 05 15:15:54 2006] [error] proxy: HTTP: disabled connection for (rchportplb.na.lennoxintl.com)
[Wed Apr 05 15:15:59 2006] [error] proxy: HTTP: disabled connection for (rchportplb.na.lennoxintl.com)
[Wed Apr 05 15:16:02 2006] [error] proxy: HTTP: disabled connection for (rchportplb.na.lennoxintl.com)
|
It appears the the worker process gets re-enabled some how after a certain time period. Do you know what the time out is and how to set it to a shorter period?
We are in the process of moving this to a Redhat Linux box and hopefully we won't see any of these problems. But, I have to make sure our production system is up and running until then so nny help is GREATLY appreciated.
Thanks,
Andrew |
|
Back to top |
|
Pulse77
Joined: 07 Jan 2008 Posts: 9 Location: Pretoria
|
Posted: Wed 09 Jan '08 15:42 Post subject: Same problem |
|
|
I just got the same error:
Faulting application httpd.exe, version 2.2.6.0, faulting module libaprutil-1.dll, version 1.2.10.0, fault address 0x00001a75.
The funny thing is that I am not using any proxy modules, and that I am running Windows 2003 R2 Standard Edition Service Pack 2.
Does the MS patch (http://support.microsoft.com/?kbid=898060) still aply to SP2 or did SP2 fix it, becuase I don't see a download for it on that page.
I Am also getting an error:
Faulting application httpd.exe, version 2.2.6.0, faulting module libapr-1.dll, version 1.2.11.0, fault address 0x00007ea8.
P.S. I am running Apache 2.2.6 from apachelounge
Thanks |
|
Back to top |
|
Pulse77
Joined: 07 Jan 2008 Posts: 9 Location: Pretoria
|
Posted: Wed 09 Jan '08 23:09 Post subject: New error |
|
|
I just spotted this new error in the event log on our production server:
Faulting application httpd.exe, version 2.2.6.0, faulting module ntdll.dll, version 5.2.3790.3959, fault address 0x0002bbec.
Anyone have any new ideas on how I can try to fix this
Thanks in advance |
|
Back to top |
|
PipoDeClown
Joined: 20 Dec 2005 Posts: 77
|
Posted: Thu 10 Jan '08 0:16 Post subject: |
|
|
- make full back
- chkdsk /f
- uninstall, reboot and reinstall the necessary vc8 runtime
- remove not used software and or tools
- stop services (database, exchange, whatever) and check if u still get the same errors
- check path settings
- check /Noexecute (i prefer setting it to Alwaysoff)
etc
- check fileaccess to log/error-files for apache service
etc... |
|
Back to top |
|