ID:65489
 
Resolved
Fixed in version 441.
BYOND Version:436
Operating System:Windows XP Home
Web Browser:Firefox 3.0.6
Status: Resolved (441)

This issue has been resolved.
Descriptive Problem Summary:
The CONTENT part of the associative list which is created by calling world.Export on an HTTP-page randomly contains unexpected bytes (2 byte hexadecimal, followed by a linefeed/carriage return, then the actual content followed by another lf/cr and a "0").
Different web-pages result in different leading hexadecimal bytes.

Numbered Steps to Reproduce Problem:
1. Compile and run the snippet
2. Repeatedly use the Send_Test_URL-verb
3. Randomly, there will be unexpected additional bytes

Code Snippet (if applicable) to Reproduce Problem:
mob
verb
Send_Test_URL()
var/http[] = world.Export("http://schnitzelnagler.farawayhost.com/ftest1.php?a=test")
if(!http)
usr << "Failed to connect."
return
var/F = http["CONTENT"]
if(F)
usr << html_encode(file2text(F))


Expected Results:
Predictable output with just the content of the website retrieved.

Actual Results:
Unpredictable output.

Does the problem occur:
Every time? Or how often? -> Random
In other games? n/a
On other computers? -> Tiberath does not seem to have this problem.
In other user accounts? ->Yes

When does the problem NOT occur? -> Never

Workarounds:
Kind of filter the content retrieved.
I'm still having an ungodly large amount of trouble reproducing this bug. But it happens when you world.Export off of my own server IIRC, so it can't be an issue limited to farawayhost.com.

It'd be nice if there was another German DM user (preferably with the same ISP) that could test this out. It's possible that it might be an issue brought up by something your ISP is doing.
Yeah, Investigated was a miss-click, by the way.
Tiberath wrote:
I'm still having an ungodly large amount of trouble reproducing this bug.

Well, I can not hand you more information than the one provided, unless you ask for something specific.
The bug persists for me, so I simply gave up on the use of world.Export.


Tiberath wrote:
But it happens when you world.Export off of my own server IIRC, so it can't be an issue limited to farawayhost.com.

Schnitzelnagler wrote in the bugreport:
Different web-pages result in different leading hexadecimal bytes.

Yes. I mentioned that and only used the farawayhost URL as an example for the code snippet, since I wanted to provide something that is "copy&paste-able".


Tiberath wrote:
It'd be nice if there was another German DM user (preferably with the same ISP) that could test this out. It's possible that it might be an issue brought up by something your ISP is doing.

As much as I wish this could be true, it would be rather not logic.
When a browser is sending a GET, the result is flawless.
I mentioned that in the posting on code problems which I thought I had linked to in my bugreport.
Might have gotten lost when converting to the tracker.
I am pretty sure the error is entirely caused by the "?a=test" part of the given URL; internally, BYOND is probably treating this as if it is the topic part of a request to a BYOND server (for what I think Export() was primarily designed for). I think the main issue is that the world is then expecting a response as if it were a response from another server, not from an HTTP request. The error also occurs with similarly formed URLs such as http://www.byond.com/developer/forum/?id=697475.

It is also worth noting that the bug report isn't entirely accurate. The response being received is in the format of:
"size
content
0"

-- Size is given in hexadecimal (and can be more or less than 2 bytes). In the given example, the printed size is "32" to indicate a numeric size of 50; this is indeed the length of the text output of the sample URL.

Edit: I think it would be pretty reasonable for someone with the ability to do so to re-list this bug as "verified." ;)
Kuraudo wrote:
I am pretty sure the error is entirely caused by the "?a=test" part of the given URL; internally, BYOND is probably treating this as if it is the topic part of a request to a BYOND server (for what I think Export() was primarily designed for). I think the main issue is that the world is then expecting a response as if it were a response from another server, not from an HTTP request. The error also occurs with similarly formed URLs such as http://www.byond.com/developer/forum/?id=697475.

To me, the error occurred for pages that do not contain a question mark as well. For example byond.com, or google.com, so unless that is internally being translated to something else, I doubt that thesis.

It would be strange that Tiberath could not reproduce the issue in this case as well.


Kuraudo wrote:
It is also worth noting that the bug report isn't entirely accurate. The response being received is in the format of:
"size
content
0"

-- Size is given in hexadecimal (and can be more or less than 2 bytes). In the given example, the printed size is "32" to indicate a numeric size of 50; this is indeed the length of the text output of the sample URL.

Actually, there are multiple (random?) appearances of this.

While I did not figure out that the leading number was indeed the size, sometimes there is a "0" in the end, or not, or even zeros in-between paragraphs.


Kuraudo wrote:
Edit: I think it would be pretty reasonable for someone with the ability to do so to re-list this bug as "verified." ;)

Thanks! ;)
Schnitzelnagler wrote:
To me, the error occurred for pages that do not contain a question mark as well. For example byond.com, or google.com, so unless that is internally being translated to something else, I doubt that thesis.

Consider my thesis retracted. It was a hunch that I based on my general understanding of world.Export()'s functionality and the uniqueness of your URL, but it also happens with http://www.byond.com for me. It's curious that Tiberath doesn't experience this.

The posted bug report was for version 436, I'm using 440.1016 so it's still a current bug.
440.1018 And the problem is non-existent for me.
Er, didn't realize that 440.1016 wasn't current. Just updated, problem still present.
The extra bytes were due to the HTTP/1.1 transfer-encoding (chunking of data), which wasn't properly being accounted for. Fixed in version 441.
The extra bytes were due to the HTTP/1.1 transfer-encoding (chunking of data), which wasn't properly being accounted for. Fixed in version 441.