Mouse Tracking

Page: 1 2
ID:1373590
 
Sep 9 2013, 6:19 pm (Edited on Sep 9 2013, 7:58 pm)
Applies to:DM Language
Status: Open
Issue hasn't been assigned a status value.
----Rekindling this conversation from several prior threads, mainly Calus CoRPS' Feature Request, mouse tracking is a very useful feature. A couple of years ago, I discovered a quirk in Internet Explorer that allowed me access to a workaround implementation. I have developed a library with it and have been using this for several years. Several BYOND developers have used the ability to track the mouse for their projects:

Faction Wars 2 by D4RK3 54BR3
Research Facility Chiron by D4RK3 54BR3
Darkness by Bravo1
Tanx by Bravo1
Keep It Up by Kaiochao
Shootah by Kaiochao
Mouse Position by Forum_account
Ultimate Jigsaw by Hiro The Dragon King
LetterBox by Hiro The Dragon King
HDK Mouse And Keys by Hiro The Dragon King

----Just to be clear, the extent of this request is any form of mouse tracking. There are several different features that I would like to come out of this though. If I forget anything, I will ammend this post.

1) Mouse coordinates in relation to the client's actual computer screen. (Used in Ultimate Jigsaw)
----- Equivalent to JavaScript's event.screenX/Y.

2) Mouse coordinates in relation to a specifc interface element, in pixels from the top-left of the element.
----- Equivalent to JavaScript's event.clientX/Y.
----- Most likely used for map elements, but who knows?

3) Mouse coordinates in relation to client.screen.
----- This would mean tiled coordinates, most likely, along with pixel offsets.
----- I would much prefer the position in pixels from the bottom-left of the client's screen.
----- This must work outside of the range of the map element itself, of course compensating for map.icon-size and map.view-size.

4) Mouse coordinates in relation to the acutal map.

----As far as implementation goes, I've been so buried in my own implementation that I seriously can't think of another way. My preference for this is to use a winset() call to trigger the mouse tracking. I was thinking that a flag of some sort for elements would be prudent. 
//1)
winset(client, null, "mouse-tracking=true")
//2)
winset(client, "element", "mouse-tracking=true")
//3)
winset(client, "map", "screen-tracking=true")
//4
winset(client, "map", "map-tracking=true")
This is a similar, yet simpler way to how my library handles it, as I have to use calls to JavaScript to start and stop tracking.

----Interacting with the data once the client starts sending it is a much more complicated matter. In my library, Topic() has to catch all of the incoming data and route it to UpdateMouse(ElementID,PositionX,PositionY). This works because my script calculates the icon-size and view-size of the map element and scales the positions accordingly, allowing me to calculate screen_loc and map coordinates on the server without a buttload of winget() calls.

----Note though, that I have not yet added support for isometric maps, and I don't know if I will. This is mostly because the math is annoying and I am butthurt that I cannot set the height of the client's view.
Sep 9 2013, 6:39 pm (Edited on Sep 9 2013, 6:45 pm)
It's difficult to determine if this is appropriate in BYOND, given the nature of BYOND's networking model.

Mouse tracking in general is okay, but controls based upon mouse tracking, like looking around or moving, actually feel clunky and unresponsive over the network for a BYOND game.

The reason for this is because there's a sort of instant feedback that's expected when mouse controls are being used. For example, if I move my mouse in Research Facility Chiron, I expect my character to immediately look towards the mouse and I expect the camera to scroll appropriately. And when it doesn't happen instantly, the network lag, no matter how little there may be, is highlighted and directly expressed to the player.
Because there's no client side processing, the game feedback from mouse tracking tends to be delayed when in a multiplayer environment.

This is why Faction Wars 2 and Research Facility Chiron never felt smooth and comfortable on BYOND, and it's actually a part of the reason why these games weren't too successful; clunky and unresponsive mouse controls are really tedious to play with and get in the way of fun.
Sep 9 2013, 6:42 pm
This could be useful in the same way that you'd winget() a window size.

winget(src,"window1","mousex")
winget(src,"window1.control1","mousey")


As long as there's a window that you can use for relativity, then mouse controls could be tracked. Even if the control was hidden, I'm sure you can still return the size and position of a window. So why not mousex, mousey?

EDIT: mousex and mousey could be concatenated to mousepos, similar to pos and size of window controls outputting "100x100" or "30,50"
Sep 9 2013, 6:43 pm
I would agree, but also add that for the more casual games like I create, the issues aren't as apparent. Just as well, BYOND can be used for single-player games.

Adding to this, the libraries FA and I made had no method of queuing mouse input or thinning that queue. This is something I think could be more easily rectified by native implementation, but I doubt it would ever allow multi-player action shooters like these to work effectively.
Sep 9 2013, 6:45 pm
In response to Makeii
Makeii wrote:
This could be useful in the same way that you'd winget() a window size.

It wouldn't, because no matter how quick your connection is, the information would -always- be late. Your method asks the client to send the server information and then waits for it.
Sep 9 2013, 6:49 pm
I would actually prefer there to be a mouseMoved sort of event that calls a function, like mouseEntered(). But only if it doesn't interrupt the input queue.

A winget() solution is kind of slow because of how the server would have to ask the client what the mouse coordinates are, and then the client has to send the mouse coordinates to the server again. So if you look at it from a game logic perspective, when the server requests mouse coordinates from a client, it could easily be more than 0.25 seconds later when it receives them from the client, EVEN when the ping time is <150ms (which is relatively good by any game's standards). And only after these 0.25 seconds will the server be capable of updating the game-state and send feedback and game-state data back to the client.

So in other words, the feedback on the mouse position experienced by the client can be delayed by more than 0.4 seconds even on a good connection.
Sep 9 2013, 6:51 pm
In response to D4RK3 54B3R
D4RK3 54B3R wrote:
I would actually prefer there to be a mouseMoved sort of event that calls a function, like mouseEntered(). But only if it doesn't interrupt the input queue.

This is essentially what my library does, as UpdateMouse() is only called if the mouse actually moves.
Sep 9 2013, 6:52 pm
In response to Hiro the Dragon King
Hiro the Dragon King wrote:
This is essentially what my library does, as UpdateMouse() is only called if the mouse actually moves.

Yes I know. I think that's the only feasible way you can do it with the current networking model.
Sep 10 2013, 12:32 pm
In response to D4RK3 54B3R
D4RK3 54B3R wrote:
I would actually prefer there to be a mouseMoved sort of event that calls a function, like mouseEntered(). But only if it doesn't interrupt the input queue.

The MouseMove() approach is in for 500, but it only applies over the non-letterboxed area of the map itself.

Not interrupting the input queue would be easiest by adding new messaging, rather than using a verb. Which, I suppose, is perfectly reasonable, though again I only see that being very useful for tracking purposes, not for general interface interaction. (There's an open question, in my mind, whether tracking would be any use without opening up more skin-friendly mouse foo in general, which is a harder thing to setup.)

I suppose as long as the program is in the foreground, sending mouse coords that we get via client-side polling wouldn't be so bad. A simple message consisting of the coords relative to the screen, relative to the main window, relative to the main window interior, in map screen coordinates, and in map tile coordinates, would probably all be useful for tracking without necessarily being overkill. This kind of thing would almost scream for var access, though, rather than a proc that gets called when the coordinates come in, and I don't fancy the idea of adding a million vars any more than Hiro likes the idea of parsing a string all the time.
Sep 10 2013, 12:47 pm
In response to Lummox JR
It's never too late to start implementing new components in the form of datums instead of prefixed variables and procedures.
Sep 10 2013, 12:56 pm
In response to Kaiochao
Kaiochao wrote:
It's never too late to start implementing new components in the form of datums instead of prefixed variables and procedures.

A datum would make some sense, except then you might end up with monstrosities like client.screen_coords.x or such. Maybe not such a big deal though.
Sep 10 2013, 1:00 pm
In response to Lummox JR
Sure, a native 2D vector datum would be great, too!
Sep 10 2013, 1:11 pm
In response to Kaiochao
If I datumized mouse coords I'd probably do so like this:

coords
  var/x, y, pixel_x, pixel_y

There are a number of internal reasons I wouldn't necessarily want to go this route with mouse tracking. For instance, if it's a normal datum, it has to be created when read from the client; it would be a snapshot of the coords, and any time it was read it would create a new datum. I did this with atom.transform because I don't think creating a special type and tying it directly to the atom is worthwhile, but with coords that might be another ball of wax entirely. The var approach may not be really suitable for mouse tracking after all; I suspect it would have to be a proc.
Sep 10 2013, 1:35 pm
In response to Lummox JR
I guess an immutable native vector datum could deserve its own thread.

I mentioned before in some other thread that using a vecX in place of X number variables ideally wouldn't be different performance-wise in any case, and the vector would still have access to useful functions and arithmetic operators. Is that even possible?
Sep 10 2013, 2:12 pm
A procedure like UpdateMouse() wouldn't be all that bad, even if I had to parse a string to do it. Something like mouse parameters wouldn't be that bad. It's not like I don't already have to do it at this point, anyway. Not to mention, processing speed won't ever really be the bottleneck with this feature anyway.
Sep 10 2013, 2:21 pm
You could use some kind of functional mouse tracker "object" for this.

For example:
client.StartMouseTracking()//Creates the mouse tracker object to start tracking.
client.StopMouseTracking()//Destroys the mouse tracker object to stop tracking.

MouseTracker
    var
        x
        y
        pixel_x
        pixel_y
    New()
        //Update the MouseTracker's vars.

What this means is that the tracker object would only be created once, to track the mouse. This also means that it can be destroyed, so you could conditionally track the mouse as needed. This may be a strange thing to have built-in, but it might work.
Sep 10 2013, 3:50 pm
In response to Kaiochao
Kaiochao wrote:
I mentioned before in some other thread that using a vecX in place of X number variables ideally wouldn't be different performance-wise in any case, and the vector would still have access to useful functions and arithmetic operators. Is that even possible?

It's possible to use vectors, but impractical. They'd have to either be tied into the original data somehow (requiring a special type), or made as a copy on the fly, which would involve relatively pointless memory allocations. With atom.transform I went with the copy method because it seemed more expedient and it's not the kind of thing you'd be reading often. With mouse vars they'd be polled on a frequent basis.
May 10 2014, 10:21 pm (Edited on May 16 2014, 12:10 am)
With the discussion of replacing IE as the native browser and the production of an HTML5 client, I'd like to once again rekindle this conversation.

I will soon be updating my library soon to follow a datum oriented approach. This is how I will organize it. I believe this solution would be the best for native implementation as well. As Multiverse7 stated, it would be quite simple to start and stop tracking the mouse.

MouseTracker
    var
        //This would identify what to track.
        id = ""

        //If ID is empty, these will be the screen coordinates,
        //the equivalent of JavaScript's event.screenX/screenY.
        //If ID refers to an element or window, it'll track in
        //relation to the upper corner of the element.
        x
        y

        //These will be the coordinates compensating for any
        //borders, titlebars (windows), letterboxing (maps),
        //or other things I can't think of.
        offset_x
        offset_y

        //If the element is a map, there are more options.
        //This first set is the equivalent of screen_loc
        //coordinates.
        tile_x
        tile_y
        pixel_x
        pixel_y

        //This set refers to the coordinates on the actual map,
        //the equivalent of atom.x/y/pixel_x/pixely.
        map_x
        map_y
        map_pixel_x
        map_pixel_y

    proc
        //Called when updating, allowing a hook.
        Update()
Feb 13 2015, 9:03 am
Bump. I'm particularly interested in mouse coordinate tracking based on relation to the clients computer screen. This will be handy for showing a context menu when right clicking items in the inventory for example.
Feb 13 2015, 9:19 am
In response to Lavitiz
In the webclient, this can be easily done using JavaScript mouse events.

In DreamSeeker, you can already get mouse coordinates relative to the map, and you've always been able to get the map's position relative to the client's screen, so just put two and two together.
Page: 1 2