I propose that the scoring system be changed so the standings of the ladder more accurately reflect the skill of the pilots.
In the past this scoring system has worked with good results. I think we should give it a shot.
This change is needed because active pilots are at the top of the ladder not the best pilots.
Scoring
Each sortie flown will award 5 points for the winner and take 5 points from the loser.
If you win all ROEs outright, (2-0, 2-0, 2-0), then you would have 30 points added to your score. While the other pilot would have 30 points subtracted from their score.
If you barely win all ROEs, (2-1, 2-1, 2-1), then you would have 15 points added to your score. While the other pilot would have 15 points subtracted from their score.
If you win all 2 ROEs, (2-1, 2-1, 1-2), then you would have 10 points added to your score. While the other pilot would have 5 points subtracted from their score.
Match wins would still be the pilot who won 2 out of 3 ROEs.
This would make climbing the ladder more challengeing and more rewarding.
Challenging and rankings
Tiered placement system on the ladder.
Similar to the one of the former Falcon 4.0 pyramid competition:
http://www.pyramidhq.com/p_grid_1024.shtml
Pilots in each ranking could challenge any pilot in the same ranking, up or down, but only the top pilots, depending on the number of pilots in the rank, could challenge the next rank.
Once a victory was achieved vs a higher ranked opponent, and the points were high enough, the pilot would acheive the next rank and then move up a rank where the process would repeat until the pilot was recycled with honors.
Please provide your input on these suggestoins.
Pilots who were in the former Falcon 4.0 pyramid competition please clarify and correct any mistakes.
SCORING SYSTEM for the RACE discussed/explained.
-
centermass
- Posts: 524
- Joined: Sun Sep 17, 2006 10:38 am
- Contact:
I'm not sure I understand. If you look at the top 6 pilots I don't think you will get an argument that they are pretty good. Season 2 has had a shakey start. With the introduction of Secure Falcon and losing the site for sometime.
Ladder seeding also didn't help, no one reached 1700 points as expected, so as new guys were entering the ladder they were started at 1500 points vice starting at the bottom of the ladder which caused the standings page to reflect inflated skill levels.
Adjustments need to be made and the standings page is still a work in progress but I'm sure we will find the right combo to make the webleague scoring system work just fine.
BUFF
Ladder seeding also didn't help, no one reached 1700 points as expected, so as new guys were entering the ladder they were started at 1500 points vice starting at the bottom of the ladder which caused the standings page to reflect inflated skill levels.
Adjustments need to be made and the standings page is still a work in progress but I'm sure we will find the right combo to make the webleague scoring system work just fine.
BUFF
-
centermass
- Posts: 524
- Joined: Sun Sep 17, 2006 10:38 am
- Contact:
-
centermass
- Posts: 524
- Joined: Sun Sep 17, 2006 10:38 am
- Contact:
-
KidVicious
- Posts: 422
- Joined: Sun Sep 17, 2006 2:50 am
- Location: Texas
- Contact:
I agree with Centermass.
Perhaps this could be considered for next season. It is clear that with the current scoring system based purely off of ROE victories that it rewards active pilots regardless of skill level.
I think what he's getting at is a 'more accurate' scoring system that reflects individual pilot skills without having the 'bug' of moving up in the competition just due to high levels of activity.
In the current scoring system, you can stay in the top % of this ladder by only winning 50% (or less) of your ROEs.
By scoring individual sorties on a point system, close matches are more closely recorded rather than ROE victory, and all out sweeps on your opponent has greater rewards for gaining points.
For example:
A match ending:
ROE1 2-1
ROE2 2-1
ROE3 0-2
....is a statistical draw (4 kills apiece). This is the closest you can get, yet the pilot that was defeated takes a full on loss regardless of how close the match was, or the fact that he shot down his opponent as many times as he was shot down.
Although it would be considered a match victory, the victor would not score any points due to being shot down an equal amount of times by his opponent.
5 points for a kill, -5 points for getting shot down is a very tried scoring system for these ROEs.
If you sweep your opponent 2-0,2-0,2-0 you are rewarded with 30 points (unless bonus points are included for a sweep, then it would be more)!
The closer the match is with your opponent, the less points you gain(or lose for that matter if you lose the match), and the more difficult it is to climb the ladder. It drives pilots to achieve a 'commanding' victory of their opponents, and reflects more closely on their overall skill level....rather than being in the top ten with a over a 50% loss rate.
With this scoring system, there are NO rewards for just flying as much as possible, you actually have to be defeating your opponents soundly to increase your score, and ascend the ladder.
Perhaps this could be considered for next season. It is clear that with the current scoring system based purely off of ROE victories that it rewards active pilots regardless of skill level.
I think what he's getting at is a 'more accurate' scoring system that reflects individual pilot skills without having the 'bug' of moving up in the competition just due to high levels of activity.
In the current scoring system, you can stay in the top % of this ladder by only winning 50% (or less) of your ROEs.
By scoring individual sorties on a point system, close matches are more closely recorded rather than ROE victory, and all out sweeps on your opponent has greater rewards for gaining points.
For example:
A match ending:
ROE1 2-1
ROE2 2-1
ROE3 0-2
....is a statistical draw (4 kills apiece). This is the closest you can get, yet the pilot that was defeated takes a full on loss regardless of how close the match was, or the fact that he shot down his opponent as many times as he was shot down.
Although it would be considered a match victory, the victor would not score any points due to being shot down an equal amount of times by his opponent.
5 points for a kill, -5 points for getting shot down is a very tried scoring system for these ROEs.
If you sweep your opponent 2-0,2-0,2-0 you are rewarded with 30 points (unless bonus points are included for a sweep, then it would be more)!
The closer the match is with your opponent, the less points you gain(or lose for that matter if you lose the match), and the more difficult it is to climb the ladder. It drives pilots to achieve a 'commanding' victory of their opponents, and reflects more closely on their overall skill level....rather than being in the top ten with a over a 50% loss rate.
With this scoring system, there are NO rewards for just flying as much as possible, you actually have to be defeating your opponents soundly to increase your score, and ascend the ladder.
Well, as a fairly new member I won't claim to fully comprehend the current scoring system, though I have a fair grasp on it. and this would certainly seem to be an improvment too it. anytime you can get a more accurate picture of skill rather than activity in a standings system it is better.
and if you want to help keep the ladder fluid and prevent it from stagnating...you have to do something about pilots on leave maintaining their positions for so long. (though this is another subject alltogether)
and if you want to help keep the ladder fluid and prevent it from stagnating...you have to do something about pilots on leave maintaining their positions for so long. (though this is another subject alltogether)
We have had a number of long discussions amongst the admins and with some of the players raising the concerns.
The main area of concern is that the Ladder does not reflect the respective skill levels of vPilots.
Other things brought up have been that the closeness of a match result isn't being taken into account in the scoring, and that higher frequency of flying contributes to the rating.
1) Ladder not reflecting relative skills.
In order to be able to respond to this concern, we have done some homework to better understand the system being used. It is called "Elo Rating".
Here is the formula that is used:
Elo Rating is a system that can be used to estimate the probability of one rated player winning against another rated player based on their respective skills. The higher your rating, the higher your skill. This system is used in National and International rankings of Chess, Table Tennis, Go, Scrabble, and other games. Please read the content of the above link if you are interested in understanding more about this system. There is a lot of statistical maths involved, so, unless you are a mathematician, don't expect to understand it on the first read through. It is also important to note that this system does not depend of any particular way of playing a game.
After reading through that document, it has become clear that we have not been managing the Web League properly. The most significant contributors to the problem have been:
1) New vPilots joining the league and being issued with a rating before determining what it should be.
2) vPilots who joined when the entry level rating was high (eg. 1500), but didn't fly any matches, (Thus some vPilots who haven't flown a match have been showing relatively high on the ladder. This has been an advantage for some who signed up a while ago but only stared flying recently and retained the rating from the past instead of being assigned a new one.) and
3) New vPilots who've joined more recently and have been assigned a progressively lower than those who joined in #2 above (This has been a disadvantage to some.)
Here are the changes that will be implemented:
1) New vPilots (and those who haven't yet flown a match) will not get a rating until they have completed three (3) trial matches. These 3 matches will be conducted as follows:
1a) The ladder will be sectioned into 3 parts. (Top, middle, lower thirds.)
1b) The new vPilot will challenge 1 vPilot from each of these 3 sections.
1c) The new vPilot will be allowed to choose who they will challenge. (Thus maxamising the randomness of the match.)
1d) The vPilots (already on the ladder) who are challenged by the new vPilot will fly this match in parallel to any other challenges they are involved. (Another way to say this is that it's an extra match on the side.)
1e) For the challenged vPilot, the result of this match will not affect their rating.
1f) For the new vPilot, they will be credited with the rating they receive from each of the matches.
1g) For the new vPilot, their entry rating into the table will be determined by averaging the 3 ratings they received from the 3 matches they have conducted.
1h) All existing vPilots who are active will receive a numerical ranking (ie. the 'Pooled' group will be used for another purpose, see below).
1h) Until the new vPilot has achieved a rating, they will show on the ladder in a "Pooled" group. (Not to be confused with the current 'Pooled' group.)
2) Closeness of a match result not being taken into account in the scoring
Each match consists of three (3) ROEs, and each ROE is scored as a seperate victory. So a victory in a match is actually three (3) individual victories.
Whilst the score in a match could be 2-1 for all three (3) ROE, since each one is scored individually each one should be considered seperately.
In the example:
ROE1 2-1 - vPilot1's rating goes up/vPilot2's rating goes down.
ROE2 2-1 - vPilot1's rating goes up again/vPilot2's rating goes down again.
ROE3 0-2 - vPilot1's rating goes down/vPilot2's rating goes up.
The amount of points gained/lost depends on the rankings of the vPilots involved at the start of the Match, and after each ROE is scored.
So, even though the closeness of a match isn't accounted for in the way that was shown using the alternate scoring system proposed, the overall effect isn't quite as great as would appear.
3) Frequency of flying contributes to the rating
The current scoring system does not directly reward the frequency of matches flown. The only aspect of a match that is scored is a win or a loss.
Thus vPilots who fly more often, unless being challenged from below, are risking their rating by challenging a vPilot above them on the ladder (and therefore with a greater skill).
Obviously a side-effect of frequent matches is that a vPilot gains more match practice which can contribute to their skill development.
4) Strengths and Weaknesses
Some vPilots are stronger in some disciplines than others. So in these situtations (depending on their opponents), these vPilots might have a tendancy of being victorious in those ROEs where they are strong, and perhaps losing in the ones where they are weak.
Since each ROE is scored as a seperate game, these vPilots are rewarded for their strengths and penalised for their weaknesses.
Obviously it is desirable to be strong in all three ROE, and one would hope that each vPilot is working on improving their weaknesses. But the urgency of this improvement is dictated by their position on the ladder and the other vPilots above/below them and is up to the individual pilot to manage.
Clearly a vPilot who is stronger in more ROEs than another pilot will take a higher place on the ladder.
Summary
Since the season is already in progress, lets see how we go with the existing system for the moment, but better managed. We will all get a better understanding of how it works by the time the season ends and can then make a decision on what steps to take.
We will implement the 'Pool' for new players ASAP.
Cheers,
Nap
The main area of concern is that the Ladder does not reflect the respective skill levels of vPilots.
Other things brought up have been that the closeness of a match result isn't being taken into account in the scoring, and that higher frequency of flying contributes to the rating.
1) Ladder not reflecting relative skills.
In order to be able to respond to this concern, we have done some homework to better understand the system being used. It is called "Elo Rating".
Here is the formula that is used:
Code: Select all
ratingoldwinner
ratingoldloser
constant = 32; (sets the maximum points that can be gained/lost)
rw1 = ratingoldwinner - ratingoldloser;
rw2 = -rw1/400;
rw3 = pow(10,rw2); (means 10 to the power of rw2)
rw4 = rw3 + 1;
rw5 = 1/rw4;
rw6 = 1 - rw5;
rw7 = constant * rw6;
rd=round(rw7); (drops the fractional part of the number)
ratingnewwinner = ratingoldwinner + rd;
ratingnewloser = ratingoldloser - rd;
Elo Rating is a system that can be used to estimate the probability of one rated player winning against another rated player based on their respective skills. The higher your rating, the higher your skill. This system is used in National and International rankings of Chess, Table Tennis, Go, Scrabble, and other games. Please read the content of the above link if you are interested in understanding more about this system. There is a lot of statistical maths involved, so, unless you are a mathematician, don't expect to understand it on the first read through. It is also important to note that this system does not depend of any particular way of playing a game.
After reading through that document, it has become clear that we have not been managing the Web League properly. The most significant contributors to the problem have been:
1) New vPilots joining the league and being issued with a rating before determining what it should be.
2) vPilots who joined when the entry level rating was high (eg. 1500), but didn't fly any matches, (Thus some vPilots who haven't flown a match have been showing relatively high on the ladder. This has been an advantage for some who signed up a while ago but only stared flying recently and retained the rating from the past instead of being assigned a new one.) and
3) New vPilots who've joined more recently and have been assigned a progressively lower than those who joined in #2 above (This has been a disadvantage to some.)
Here are the changes that will be implemented:
1) New vPilots (and those who haven't yet flown a match) will not get a rating until they have completed three (3) trial matches. These 3 matches will be conducted as follows:
1a) The ladder will be sectioned into 3 parts. (Top, middle, lower thirds.)
1b) The new vPilot will challenge 1 vPilot from each of these 3 sections.
1c) The new vPilot will be allowed to choose who they will challenge. (Thus maxamising the randomness of the match.)
1d) The vPilots (already on the ladder) who are challenged by the new vPilot will fly this match in parallel to any other challenges they are involved. (Another way to say this is that it's an extra match on the side.)
1e) For the challenged vPilot, the result of this match will not affect their rating.
1f) For the new vPilot, they will be credited with the rating they receive from each of the matches.
1g) For the new vPilot, their entry rating into the table will be determined by averaging the 3 ratings they received from the 3 matches they have conducted.
1h) All existing vPilots who are active will receive a numerical ranking (ie. the 'Pooled' group will be used for another purpose, see below).
1h) Until the new vPilot has achieved a rating, they will show on the ladder in a "Pooled" group. (Not to be confused with the current 'Pooled' group.)
2) Closeness of a match result not being taken into account in the scoring
Each match consists of three (3) ROEs, and each ROE is scored as a seperate victory. So a victory in a match is actually three (3) individual victories.
Whilst the score in a match could be 2-1 for all three (3) ROE, since each one is scored individually each one should be considered seperately.
In the example:
ROE1 2-1 - vPilot1's rating goes up/vPilot2's rating goes down.
ROE2 2-1 - vPilot1's rating goes up again/vPilot2's rating goes down again.
ROE3 0-2 - vPilot1's rating goes down/vPilot2's rating goes up.
The amount of points gained/lost depends on the rankings of the vPilots involved at the start of the Match, and after each ROE is scored.
So, even though the closeness of a match isn't accounted for in the way that was shown using the alternate scoring system proposed, the overall effect isn't quite as great as would appear.
3) Frequency of flying contributes to the rating
The current scoring system does not directly reward the frequency of matches flown. The only aspect of a match that is scored is a win or a loss.
Thus vPilots who fly more often, unless being challenged from below, are risking their rating by challenging a vPilot above them on the ladder (and therefore with a greater skill).
Obviously a side-effect of frequent matches is that a vPilot gains more match practice which can contribute to their skill development.
4) Strengths and Weaknesses
Some vPilots are stronger in some disciplines than others. So in these situtations (depending on their opponents), these vPilots might have a tendancy of being victorious in those ROEs where they are strong, and perhaps losing in the ones where they are weak.
Since each ROE is scored as a seperate game, these vPilots are rewarded for their strengths and penalised for their weaknesses.
Obviously it is desirable to be strong in all three ROE, and one would hope that each vPilot is working on improving their weaknesses. But the urgency of this improvement is dictated by their position on the ladder and the other vPilots above/below them and is up to the individual pilot to manage.
Clearly a vPilot who is stronger in more ROEs than another pilot will take a higher place on the ladder.
Summary
Since the season is already in progress, lets see how we go with the existing system for the moment, but better managed. We will all get a better understanding of how it works by the time the season ends and can then make a decision on what steps to take.
We will implement the 'Pool' for new players ASAP.
Cheers,
Nap
Last edited by Nap on Mon Oct 23, 2006 10:56 pm, edited 3 times in total.
As some people have asked about the forumla and how it works, I've created a table that you can have a look at to see how many points are awarded per game depending on who plays. (Real sample data was used to create the table.)
You can look at the table here.
Notes:
It is possible to alter the formula to change the weighting of a win/loss in various circumstances. Ie, it looks from the table that P46 would need 12 wins (ie 4 Matches) against P1 to climb up to 1st place. (Though our rules don't allow P46 to challenge P1.)
If anyone is interested in the excel spreadsheet with the formulas, let me know.
Cheers,
Nap
You can look at the table here.
Notes:
- The number shown for the Winner/Loser combination is the amount of points that will be added to the winner's rating and subtracted from the loser's rating. (Notice that it is the same amount.),
- You cannot play against yourself, therefore the empty cells,
- Notice that:
- P46 gets (27) points for beating P1, while
- P2 gets (16) for beating P1.
- Notice that:
- P1 gets (15) points for beating P2, while
- P1 gets (4) points for beating P46.
- Notice that if a new player with a rating of 0 (eg P47)(though we are not currently considering this viable since new players start at a rating one (1) point less than the lowest rated player)
- Beats a rated player, 31 points is awarded/lost,
- Looses to a rated player, the rated player does not loose any points and the new player stays on 0 rating.
It is possible to alter the formula to change the weighting of a win/loss in various circumstances. Ie, it looks from the table that P46 would need 12 wins (ie 4 Matches) against P1 to climb up to 1st place. (Though our rules don't allow P46 to challenge P1.)
If anyone is interested in the excel spreadsheet with the formulas, let me know.
Cheers,
Nap
-
centermass
- Posts: 524
- Joined: Sun Sep 17, 2006 10:38 am
- Contact:
Still seems inconsistent.
I'm not 100% sure how it works but...
I lost to Buff, all three ROEs, and went from 6th to 14th. he was 5 places behind me, not sure of the rating.
I beat Cyborg, all three ROEs, and jumped from 14th back to 6th. Cyborg was 1 spot above me and only 2 points above me.
Seems like I would not have jumped that far ahead by beating Cyborg since he was not that far ahead of me in rating.
I'm not 100% sure how it works but...
I lost to Buff, all three ROEs, and went from 6th to 14th. he was 5 places behind me, not sure of the rating.
I beat Cyborg, all three ROEs, and jumped from 14th back to 6th. Cyborg was 1 spot above me and only 2 points above me.
Seems like I would not have jumped that far ahead by beating Cyborg since he was not that far ahead of me in rating.
CM,
It's a formula, so how can it be inconsistant?
If you don't know how the formula works, that can explain why you see things you didn't expect.
For your next match, write down your rating and that of your opponent before the match.
Then, when you've finished the match, use the forumla above based on whatever the outcome was (hopefully it's all victories) to calculate the interm and final ratings.
Calculate it for ROE1, ROE2 and then ROE3. Each ROE is treated as a seperate game. (I have a spreadsheet I can send you if you like with the formula in it. Send me an ICQ and I will explain it.)
You will then see how the final rating was worked out.
Cheers,
Nap
It's a formula, so how can it be inconsistant?
If you don't know how the formula works, that can explain why you see things you didn't expect.
For your next match, write down your rating and that of your opponent before the match.
Then, when you've finished the match, use the forumla above based on whatever the outcome was (hopefully it's all victories) to calculate the interm and final ratings.
Calculate it for ROE1, ROE2 and then ROE3. Each ROE is treated as a seperate game. (I have a spreadsheet I can send you if you like with the formula in it. Send me an ICQ and I will explain it.)
You will then see how the final rating was worked out.
Cheers,
Nap
I agree with Nap and Buff for the simple reason that the current system takes into account the ranking of the guy you are flying against when scoring...(see Nap's explanation above)
Also, the only way you can have as many losses as I do and end up on the top of the ladder is if you put together a string of victories against highly ranked pilots. Thus, the moral of the story is... the pilot that losses, but does not give up, and keeps improving, challenging up and winning, will be rewarded when he starts consistently winning matches consecutively.
Also, the reason Tank went to the top in 7 games or so is because he was the last pilot grandfathered in under the rule starting new pilots at 1500, otherwise, he would have been in the same boat as KV, and had to start at the bottom.
As for the simple scoring system you suggest, one major problem with that, is stagnation at the top. If No. 1, has 2 or 3 victories winning 2-0, 2-0, 2-0, it would make it damn near impossible to ovetake him if your coming from No.15. In fact, I think under your suggested system, the reality would be, after a few weeks, the top 10's score would be out of reach for most of the other pilots. Also, whats the motivation to challenge Tank or KV if you are going to body slam your ranking if you get shut out. Shit, it would become damn boring for both of them.
Another major problem is the penalty for guys who challenge up as high as they can (4 or 5 up). Currently, if you do that and win, you are properly rewarded for pushing yourself if you win in an upset, and your ranking is not destroyed if you loose. However, under your system, the smart guy would only challenge up 1 or 2 pilots, so that he could get the points and fly in the easiest match possible. So why push yourself to be better, he should take the easiest match available, and run with the points like a little pussy ! IMHO this league should reward people who have BIG BALLS and who challenge themselves to be the best they can be.
Just my 2 cents
Also, the only way you can have as many losses as I do and end up on the top of the ladder is if you put together a string of victories against highly ranked pilots. Thus, the moral of the story is... the pilot that losses, but does not give up, and keeps improving, challenging up and winning, will be rewarded when he starts consistently winning matches consecutively.
Also, the reason Tank went to the top in 7 games or so is because he was the last pilot grandfathered in under the rule starting new pilots at 1500, otherwise, he would have been in the same boat as KV, and had to start at the bottom.
As for the simple scoring system you suggest, one major problem with that, is stagnation at the top. If No. 1, has 2 or 3 victories winning 2-0, 2-0, 2-0, it would make it damn near impossible to ovetake him if your coming from No.15. In fact, I think under your suggested system, the reality would be, after a few weeks, the top 10's score would be out of reach for most of the other pilots. Also, whats the motivation to challenge Tank or KV if you are going to body slam your ranking if you get shut out. Shit, it would become damn boring for both of them.
Another major problem is the penalty for guys who challenge up as high as they can (4 or 5 up). Currently, if you do that and win, you are properly rewarded for pushing yourself if you win in an upset, and your ranking is not destroyed if you loose. However, under your system, the smart guy would only challenge up 1 or 2 pilots, so that he could get the points and fly in the easiest match possible. So why push yourself to be better, he should take the easiest match available, and run with the points like a little pussy ! IMHO this league should reward people who have BIG BALLS and who challenge themselves to be the best they can be.
Just my 2 cents
Last edited by Soprano on Tue Oct 24, 2006 10:12 pm, edited 1 time in total.
-
KidVicious
- Posts: 422
- Joined: Sun Sep 17, 2006 2:50 am
- Location: Texas
- Contact:
Who is online
Users browsing this forum: No registered users and 7 guests