HTTP Protocol Client and PHP

[Zorm]

Looks to me like it might be related to chunked transfer encoding or something fun like that. Note how the length ('d', 'e', '0') is changing for people. Why its actually 'd' and '0' I have no clue...

[Barabajagal]

If you request larger data chunks (like index.php for example), the values change. I don't really care exactly what it is, I just want to remove it from the final result (which requires accurate detection of it).

[Zorm]

Uh huh, looks like its pretty clear to me. Its the length of the data following it. So you read the first line of the first data section, get the length, read that much more. Once you have read length you repeat the process until length == 0.

[iago]

Zorm said exactly what I was thinking, that it's the length. I didn't think of that till later, though.

It may be helpful to read the RFC for HTTP. That'll explain everything you need to know to properly implement HTTP 1.1. As long as you're guessing at the implementation, you run the risk of having hard-to-find bugs like this one.

[Barabajagal]

http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1 ? I suppose that's what it is. I should have guessed "d" would have been the length, but I'm so used to binary protocols, I was looking at the byte values, not the characters.

[Camel]

HTTP is not a binary protocol. It's not guaranteed to be binary-safe, because it was implemented for 6/7-bit modems.

Why do you care about HTTP 1.1? Do you need any of the features that aren't available in 1.0?

[Barabajagal]

I know it's not binary. I said I'm used to binary. It's fixed and everything now. I needed 1.1 for the No-Cache option.

[Camel]

HTTP 1.0 doesn't explicitly support caching at all. Besides, caching is a client-side feature, not server-side. Since you're writing the client, you can just ignore the cache header.

[Barabajagal]

Doesn't matter cause it works. I've known servers that do cache copies, though.

[Camel]

Servers can't cache copies. HTTP caching means that the client won't re-download files it already has.

[Barabajagal]

I'd still like to use 1.1 (and am), so it doesn't matter.

[Michael]

do you plan on releasing this control when your done, i'd be happy to give it a try.

[Barabajagal]

I'd rather not release it separately. If you want it, download the source code to my bot when I release 1.3.1.

[Banana fanna fo fanna]

You could try the accept-encoding header, but I don't know if it supports transfer encodings. I'd suggest either fully implementing HTTP 1.1, or using HTTP 1.0.

[Barabajagal]

I already fixed the problem!!!!!
The accept-encoding header is set to *, as is language and charset. I download more than just text with this control.