Getting data from external websites in Developer Help

ID:149157
 
Jun 25 2002, 6:39 pm (Edited on Jun 25 2002, 6:45 pm)
Is there a way to get the output of an external web page or file, and set a variable to its contents?

I can send it to be viewed in the internal browser using link(), and I can read a file that is local in several different fashions, but not one off of a web server.

Export and Import are not doing it for me. They seem to expect a specific file format, but I'm just trying to deal with text. I'm not sure, but they also seem to expect the external server to be hosting a running byond world, which I can't do on the server I will be using.

Browse() also does not do what I need, as it won't take an external site as an argument.

I'm also looking for a way to send an http request to a server without displaying the results in the browser.

In case you're curious, I'm trying to develop my saved rating systems for chess, and want to use a ColdFusion server to save and retrieve scores. So any other suggestions would be helpful here for that.
#1 Jun 25 2002, 7:25 pm
In the htmllib there's a GetHtml() (or something like that) proc, you may want to look over Dantom's code to see how they did it.
#2 Jun 25 2002, 7:35 pm
In response to Nadrew (#1)
Nadrew wrote:
In the htmllib there's a GetHtml() (or something like that) proc, you may want to look over Dantom's code to see how they did it.

That library uses byond CGI. I can't run that on the server I have access to.

Basically, I just need this syntax to work:

var/myvar
link("http://www.asite.com") >> myvar

If I can send it to a user, why can't I send it to a variable?
#3 Jun 25 2002, 9:10 pm
Skysaw wrote:
Is there a way to get the output of an external web page or file, and set a variable to its contents?
[snip]
Export and Import are not doing it for me. They seem to expect a specific file format, but I'm just trying to deal with text. I'm not sure, but they also seem to expect the external server to be hosting a running byond world, which I can't do on the server I will be using.

According to the documentation, world.Export() should actually be working for this with ordinary Web sites. If it doesn't, then it's probably broken.

Lummox JR
#4 Jun 25 2002, 9:23 pm
In response to Lummox JR (#3)
Lummox JR wrote:
Skysaw wrote:
Is there a way to get the output of an external web page or file, and set a variable to its contents?
[snip]
Export and Import are not doing it for me. They seem to expect a specific file format, but I'm just trying to deal with text. I'm not sure, but they also seem to expect the external server to be hosting a running byond world, which I can't do on the server I will be using.

According to the documentation, world.Export() should actually be working for this with ordinary Web sites. If it doesn't, then it's probably broken.

Lummox JR

Actually, I don't think it was intended to do what I'm trying here. I'm not trying to export a file at all, just send an http request to a website. That much, I've actually figured out a klugy way of doing.

But as to getting information from the external site, there doesn't seem to be a way, except to pass it to the browser. I'm basically trying to parse a returned page for variables, as opposed to having byond download a file from the site.
#5 Jun 26 2002, 6:38 am
In response to Skysaw (#4)
Skysaw wrote:
Actually, I don't think it was intended to do what I'm trying here. I'm not trying to export a file at all, just send an http request to a website. That much, I've actually figured out a klugy way of doing.

You should be able to use Export() to access an external site without sending a file; sending a file is optional.

Lummox JR
#6 Jun 26 2002, 8:19 am
In response to Lummox JR (#5)
Lummox JR wrote:
Skysaw wrote:
Actually, I don't think it was intended to do what I'm trying here. I'm not trying to export a file at all, just send an http request to a website. That much, I've actually figured out a klugy way of doing.

You should be able to use Export() to access an external site without sending a file; sending a file is optional.

Lummox JR

As I mentioned, this part I can get around. However, I don't see this working for Import getting information off of a web page.
#7 Jun 26 2002, 8:33 am
In response to Skysaw (#6)
Skysaw wrote:
As I mentioned, this part I can get around. However, I don't see this working for Import getting information off of a web page.

I'm not sure I follow you; it's world.Export() that's used to access sites, not world.Import(). This is the example cited in the reference:
mob/verb/test()
  var/http[] = world.Export("http://www.byond.com")

  if(!http)
    usr << "Failed to connect."
    return

  usr << "HTTP Header:"
  for(var/V in http)
    usr << "[V] = [http[V]]"

  usr << "\n"

  var/F = http["CONTENT"]
  if(F)
    usr << html_encode(file2text(F))

I can't try this from here, but it does seem to me that if this doesn't work, a BYOND bug is responsible.

Lummox JR
#8 Jun 26 2002, 8:51 am
In response to Lummox JR (#7)
Lummox JR wrote:
Skysaw wrote:
As I mentioned, this part I can get around. However, I don't see this working for Import getting information off of a web page.

I'm not sure I follow you; it's world.Export() that's used to access sites, not world.Import(). This is the example cited in the reference:
mob/verb/test()
>   var/http[] = world.Export("http://www.byond.com")
> 
>   if(!http)
>     usr << "Failed to connect."
>     return
> 
>   usr << "HTTP Header:"
>   for(var/V in http)
>     usr << "[V] = [http[V]]"
> 
>   usr << "\n"
> 
>   var/F = http["CONTENT"]
>   if(F)
>     usr << html_encode(file2text(F))

I can't try this from here, but it does seem to me that if this doesn't work, a BYOND bug is responsible.

That looks pretty similar to one of my attempts last night, which didn't work. I'll try this approach one more time when I get a chance, since I may have missed something here.
#9 Jun 26 2002, 12:07 pm
In response to Lummox JR (#7)
Had a chance to try your code. I didn't realize that export gave a list.

But here's a new problem, which I'll repost in the bug forum. Your code works, but only for pages at byond.com. For any other site, it crashes DS.