ID:138321
 
I have no excuse, but when I encounter mathematically-related stuff my brain runs away screaming and I am at a loss. Surely this is just childhood conditioning, since the programming tasks I engage in are more complicated than the typical math problem.

Nonetheless, it's true...

So to try and understand how MANTRA works, I showed it to my partner, who has several degrees and makes a living doing statistical analysis for political campaigns. I asked him if he could tell me how this works, and here are his comments, which I offer to help push forward the MANTRA system, not to criticize it:

He feels that for the ranking system to matter to players, they have to understand it. Which implies that we should provide a description of the formula that will make sense to non-programmers.

Being a non-programmer, he couldn't understand the formula because there is no explanation of the terms used. For example, s[B][A] didn't mean anything to him, nor did r[A].

I assume after thinking about it for a while that s means score and that [B][A] is the total number of points B has accumulated during games against A. I also assume that r means ranking. Whatever the case, it would be good to provide some textual description of these, and perhaps a mathematical formula that doesn't use programming terminology like == so that our players (and my partner!) will understand.

His final comment was that, though he couldn't read the formula, it seemed that cheating would be no more complicated than creating 2 or 3 dummy identities and playing them against each other to build up a realistic profile.

Additional questions from me:

Does this formula apply to games with more than two players?

Would it make sense to do something similar for single-player that might be more expressive than total accumulated score or average score per game?

On 10/30/00 5:42 pm Deadron wrote:
Being a non-programmer, he couldn't understand the formula because there is no explanation of the terms used. For example, s[B][A] didn't mean anything to him, nor did r[A].

Reading the docs again, now I see that these terms are explained...so maybe this is more of me being a mathematical idiot.

The other questions remain though.

I have no excuse, but when I encounter mathematically-related stuff my brain runs away screaming and I am at a total loss.

Heh. I never intended that people would have to understand the algorithm in order to understand the result. A rank is something anyone can understand. Most chess players do not know the precise formula used to generate the rating numbers, but they still understand that a higher number is better than a lower one.

My main interest in publishing the algorithm was to get some helpful feedback as to whether people who do understand it agree that it indeed produces what people commonly mean by the term 'rank'.

His final comment was that, though he couldn't read the formula, it seemed
that cheating would be no more complicated than creating 2 or 3 dummy
identities and playing them against each other to build up a realistic
profile.

And he is correct! I call this, the stacked dummy attack. I decided not to advertise that feature too loudly, because I was curious how apparent it would be.

There are a couple work-arounds to avoid stacked dummies. Keep in mind, however, that in a fairly well-supported rank list, you would have to do a lot of work just to pull it off, because you would have to build up dummies in each rank to boost you to the next higher level. A simple requirement
that you cannot advance without having played somebody already in the next level would go a long way towards deflating the dummy stack! I wanted to avoid that, because then you might have higher-ranking people refusing to play lower ranking people just to keep them down.

One rather practical addition is to charge a small membership for entry into the rank list. But then you might get worried that wealthier people would be able to buy rank (by buying memberships for all of their dummies). It would
have to be a pretty serious game for that to be a problem! (And you would be soaking the wealthy egotists in the process.)

When I get a chance, I'll do some more thinking and see if there isn't a nice mathematical solution. Ultimately, you can't prevent dummies beyond the shadow of a doubt. But you can at least detect when there are disjoint groups in the score matrix. (The dummies would form a group with no ties to the rest of the database.)

Does this formula apply to games with more than two players?

Yes. That's where the "adaptable" part comes in:)

It just makes an entry in the score matrix for each pairing of players in the game. The assumption, however, is that the score of player A in a given game is the same against all the other players. That works for most types of games.

Would it make sense to do something similar for single-player that might be
more expressive than total accumulated score or average score per game?

With single player games, the appropriate point of comparison really depends on the game. Total accumulated score is probably almost never of interest. We could make the specific function a parameter of some sort.
In response to Dan
On 10/31/00 10:59 pm Dan wrote:
Heh. I never intended that people would have to understand the algorithm in order to understand the result. A rank is something anyone can understand. Most chess players do not know the precise formula used to generate the rating numbers, but they still understand that a higher number is better than a lower one.

A good point.


My main interest in publishing the algorithm was to get some helpful feedback as to whether people who do understand it agree that it indeed produces what people commonly mean by the term 'rank'.

Well I don't understand so I'll have to leave it to others!
In response to Deadron

Well I don't understand so I'll have to leave it to others!

I was writing in a state of severe sleep deprivation. Now, with a little more verve in my fingers, I will take a second shot at it.

A full explanation of almost any algorithm can leave one a bit lost in the details, when it is actually not very complicated.

The data that we have to work with, for the purpose of establishing some sort of comparison between players, is simply the record of all games played by each player against every other player. That is all I mean by a 'score matrix'. It is just a record of each player's wins and losses against the other players. Simple.

All any ranking algorithm can do is look at this score data and decide how to order the players. That would be easy if there was always a clear-cut heirarchy in which the best player never loses, the second best player only loses to the best player, and so on. The trouble is, that ain't the way the world works.

All MANTRA does is find a way of ordering the players that comes as close as possible to the ideal, in which higher ranked players always beat lower ranked ones. It does that the most brainless way (with a few optimizations): it simply tries every possible ordering and picks the best one. By 'best', I mean the one with the minimum number of conflicts (lower ranking players who beat higher ranking ones).

I think anybody can understand that. When you have to write down a mathematical formula for measuring how close a given rank set comes to the goal, then it becomes a little more foreboding, but even then, if you have a clear picture in mind of what is being accomplished, I think it is not so difficult to understand.

Perhaps I need to provide a bit more of an overview in the MANTRA docs before jumping into the math. Now that I know someone has even taking the trouble to glance through it, I can more easily muster the necessary motivation to do so!

--Dan
In response to Dan
A thought...does the owner of the game node on the hub have the ability to clear the cache of scores? That will be necessary once in a while, especially when going between Beta and Final releases.

I was going to look at the API online to see, but a look in the Libraries area doesn't show the MANTRA link.

Where is that again?
In response to Deadron
they still understand that a higher number is better than a lower one.

Not in golf, though. ;-)

Well I don't understand so I'll have to leave it to others!

What is getting past me is why Dan doesn't try doing IP checks (client.address) and comparing them between two opponents on at the same time. It might not stop dummies in e-mail games, but it'd stop dummies in realtime games (...dummies being the players, not their fake counterparts ;-). client.address does report their IP address, doesn't it?
In response to Deadron

A thought...does the owner of the game node on the hub have the ability to clear the cache of scores?

I anticipated this request! Yes. You just go in and delete the "ranks" directory. That purges the old score file.

No doubt, a time will come when you will need even finer control, like the ability to suppress a specific player or something like that. When the time comes, it shouldn't be too hard to implement additional administrative controls.


I was going to look at the API online to see, but a look in the Libraries area doesn't show the MANTRA link.

Where is that again?

There is a link to that page from inside the general hublib page. But the information in there is just the abstract algorithm. The hublib API is not well documented anywhere yet. I just tried to provide enough material to get hub games up and running on the first round.