Apache :: Odd Header Behavior

iamwyza · Joined: 16 May 2007 Posts: 15

running Server: Apache/2.0.55 (Unix) mod_ssl/2.0.55 OpenSSL/0.9.8a mod_perl/2.0.2 Perl/v5.8.8

We get an intermittent error where a random letter/number combination will be inserted between the response headers and the page header.

tdonovan

This appears to be chunked encoding.
The 1db1 indicates that the follwing section of the message body is 7,601 bytes long (7601 decimal = 1db1 hexadecimal).

The problem is that there is no:

Transfer-Encoding: chunked

header to indicate that chunked encoding is being used.

Your Apache and mod_perl are versions from 2005.
You might try more recent versions of Apache (version 2.0.59 was released July 28, 2006) and mod_perl (version 2.0.3 was released November 28, 2006).

-tom-

iamwyza · Joined: 16 May 2007 Posts: 15

Where is chunked-encoding usually defined in the server config?

tdonovan

Chunked encoding is not defined or controlled in the server config.

It is automatic (per RFC2616) if the client browser supports HTTP 1.1 and uses KeepAlive.
Most modern browsers do.

You might try adjusting your KeepAlive parameters.

If this problem always happens on the 100th request from a browser, you could set MaxKeepAliveRequests to 0 (which means "unlimited").
It is currently 100 - which is the default if you do not specify it.

Another one to try is KeepAliveTimeout . The Apache 2.0 default is 15 seconds, but a value like 5 may work better for you.

-tom-

p.s. Once you can freely choose your version of Apache, I suggest you move to Apache 2.2.
The current release, 2.2.4, is surprisingly stable with heavy volume.
The downside is that all modules must be re-built for Apache 2.2 vs. 2.0.

iamwyza · Joined: 16 May 2007 Posts: 15

when i change the first display lines from:

tdonovan

When you have Transfer-Encoding: chunked, you need the hex length (like 1db1). Chunked encoding won't work without it.

Looking at the source code for Apache 2.0.55 - it seems impossible for Apache to perform chunked-encoding without sending the Transfer-Encoding: chunked header.

Does your provider supply an altered version of Apache 2.0.55?

Another possibility is that they perform the chunked encoding in Perl (vs. letting Apache do it) and neglect to add the header.
This seems pretty unlikely though!

FYI - If you look at the Apache 2.0.55 source code for http_protocol.c:

Transfer-Encoding

(and any Content-Length header is removed)

It's hard to see how you ever get chunked encoding without the header.
The value of r->chunked is not changed between lines 1667 and 1753.

Very curious! Maybe I'm missing something. Does your Apache config use any Proxy... directives?

-tom-

iamwyza · Joined: 16 May 2007 Posts: 15

Heh, oddly enough the transfer encoding has been chunked all along, the program i was using to get the header information out conveniently was stripping it out. So no idea then why it happens. In any case i have changed the keepalive stuff and we will see if that cleans up the problem.

Thanks for your time.

tdonovan

No sweat! An interesting problem.

If changing the two directives doesn't work, you could turn KeepAlives (and therefore chunked encoding) off completely with:

iamwyza · Joined: 16 May 2007 Posts: 15

I guess then i will have to try turning keepalive off at this point because it has appeared again. Interestingly enough view source does not show the chunk numbers even though they are displayed on the non-rendered page.

tdonovan

Correct - View Source does not show the chunks - they have already been reassembled. It does not show headers either.

The question is: when you get the page which is not rendered (a blank white page?) - what does View Source show? Anything?

-tom-

iamwyza · Joined: 16 May 2007 Posts: 15

Oh, no when i say not rendered i mean the browser (any browser) doesn't turn the html into something pretty. All lines of code are printed on the screen in plain text, view source shows the same as on the screen minus the http header information and the chunk information. Sorry for the confusion.

tdonovan

Maybe you are missing the Content-Type header when this happens, and the browser is mis-interpreting the response as plain text instead of text/html.

In FireFox, you can use Tools Page-Info to check whether the Content-Type is really text/html.

iamwyza · Joined: 16 May 2007 Posts: 15

Well that makes no sense when the header that is printed plainly says Content-Type: text/html. note that when the page doesn't render the page-info says the type is text/plain. The thing i don't understand is why its intermittent. If i refresh a page that has this problem then it reloads fine and renders fine.

Here is a copy of exactly what prints to the screen when this happens.

https://cars.bethelcollege.edu/huh.txt

tdonovan

Anything which prints to the browser screen is not considered a header.
I suspect if you examine Tools Page-Info you will find that the browser thinks it is plain text, not text/html. EDIT: ooops! you just said that, didn't you.

The browser is apparently getting a blank line ahead of the "HTTP/1.1 200 OK" line
- so "HTTP/1.1 200 OK" and everything after it are not recognized as headers, but are incorrectly treated as the body of the response.

This shouldn't happen. The "HTTP/1.1 200 OK" line should be the very first thing the browser receives. No blank lines should precede it.

Do you have any 3rd-party modules installed, other than mod_perl?
If so, you could try temporarily disabling them.

If you use mod_deflate, it may be worth commenting that out too.

-tom-

EDIT: Just a thought - are you using FireFox 2.0.0.3? Does this happen with other browsers?
There's a small possibility this might be a browser bug, not an Apache bug.
I don't see any relevant Closed Apache bugs for Apache 2.0

iamwyza · Joined: 16 May 2007 Posts: 15

I found a plugin for firefox that lets me capture all of the headers sent and recieved:
below is an sdiff of them. On the left is the one that fails and on the right the one that works. There are no differences on the page or how i arrived at the page. You may have to copy it into notepad for it to format right though.

tdonovan

One important thing - posting the Authorization: header is not a good idea.
Basic Authentication is not resistant to network evesdropping (or public posting).

You should change the password for wyzaj immediately.

tdonovan

The "HTTP/1.x 200 OK" line is funny.
It should be "HTTP/1.1 200 OK".

Any idea where that 1.x comes from?
I don't think Apache 2.0.55 can generate non-numeric protocol version numbers like this.
Maybe something in your application? A grep for '1.x' might turn something up.

iamwyza · Joined: 16 May 2007 Posts: 15

though this line shows up in many places and would take a long time to go through them all. the file src/httpd-2.0.55/modules/http/http_protocol.c in the source did have this...

tdonovan

Yes - Apache only has "1.x" in comments, like the one you found in http_protocol.c.
The only two "200 OK" responses that Apache can generate are:

-and-

I meant, does your application have "1.x" in it anywhere? In a .pl script perhaps?

iamwyza · Joined: 16 May 2007 Posts: 15

I don't think so, i grepped all the code. Also every image, css, ect. are all served with http/1.x as well. The get statement uses 1.1 but the response is 1.x.