OOC:Comparison of algorithms

When scorinating certain sports, particularly association football (or soccer), using xkoranate, there is often debate over which formula (or paradigm, or algorithm) to use.

For many users, particularly newer ones, the differences between these formulae can be quite opaque. This article aims to clarify some of the more pertinent points.

Warning: There will be a considerable amount of mathematics in this article, though I have tried to simplify as much as possible.

Comparison

Fundamentally, the main difference between the formulae is how they treat rank (or "skill", as xkoranate calls it). At a basic level, NSFS uses the absolute difference between the provided values, while SQIS uses the ratio of the involved values.

The most pertinent concern most people therefore have is "how do the differences between the formulae affect results?"

Let us consider the following ranking points (or skill values) for four teams: Alpha (20), Bravo (10), Charlie (10), Delta (5).

Then let us consider two hypothetical matches: Alpha v Bravo and Charlie v Delta.

Under SQIS, the probability of the lower ranked side winning in either game is equal, because win probability is calculated based on the ratio between the competing teams' skill values, and 20:10 is the same as 10:5.

Under NSFS, the probability of a win is calculated (partly) based on the difference between the ranks. So, because (20-10) is a larger value than (10-5), there is a lower chance of Bravo beating Alpha when compared to the chance of Delta beating Charlie; since there is a larger difference between the teams' ranks.

Max Rank Effect

When using xkoranate, the value of "Maximum skill", or Max Rank (and to a lesser extent, the "Minimum skill", or Min Rank) will affect how results are calculated, regardless of the formula chosen.

This is because xkoranate makes an adjustment to ranking points ahead of running the scorination algorithm. Xkoranate's adjustRank function is intended to convert all ranks to a value between 0 and 1. It does this by taking the rank value minus the min rank, divided by the difference between min and max rank.

$Rank={\frac {(teamRank-minRank)}{(maxRank-minRank)}}$

These adjusted ranks are then used directly in the formula to determine when attacks are successful (and in the case of NSFS, the number of attacks a team gets).

So in simple terms, when using NSFS, and assuming you leave min rank at zero, upsets are more likely between lower ranked teams. By extension, a higher max rank makes all rank differences smaller, and therefore increases the range of ranks that fall into that category of "lower ranked teams". This is why it is generally recommended not to use a scale that goes from 0-100, but rather 40-100 when scorinating domestic leagues.

This conclusion is explained in greater depth in the case studies below.

Case studies

Imagine the highest ranked team in a competition has a rank of 100. They face a team ranked 10. Max rank is of course set to 100, and minimum is 0. The rank difference used in the NSFS calculation is therefore $(100-0)/(100-0)-(10-0)/(100-0)$, which simplifies to $(100-10)/(100-0)=0.9$

Going back to our original example though: $(20-10)/(20-0)=0.5$, but $(10-5)/(20-0)=0.25$

As max rank increases, assuming everything else stays the same, these rank differences get smaller (because the denominator gets larger).

Max rank too high

If max rank is larger than the highest ranked team, all of the calculated rank differences decrease.

If max rank is double that of the highest ranked team, the maximum possible rank difference becomes $((n-0)/2n)$, instead of $((n-0)/n)$, which is twice as large.

So again, back to our original example: if max rank was 100, those rank differences become $(20-10)/(100-0)=0.1$ and $(10-5)/(100-0)=0.05$.

Max rank too low

If a team has a rank higher than max rank you get the opposite effect and the rank difference they can achieve is larger.

For example, a team ranked 120 when max rank is 100, against a team ranked 10. The rank difference becomes $(120-10)/(100-0)=1.1$.

This can skew things in the calculation because NSFS expects a value between 0 and 1 when calculating rank difference. This is why you'll see teams with rank higher than max sometimes getting crazy results and blowout wins etc (depending on how far above max rank they are, and what event you're scorinating).

Other Issues

NSFS Home Advantage

In xkoranate 0.3.3 home advantage is not applied correctly in the NSFS paradigm. It should apply a multiplier to the denominator of the probability calculation, but instead simply adds 1 (the parsed integer value of a "true" boolean).

How it works is that the probability of a goal, or P(goal) is calculated as:

$1-(baseAttackSuccessThreshold+rankDiffModifier*((oppSkill*rankCoeff)^{r}-(skill*rankCoeff)^{r}))/(baseAttackCoeff+homeAdvantage))$

where r = rankScalar (by default this is 0.5, or square root)

The example calculation below uses the default config values for the NSFS paradigm, and two teams of equal skill:

$1-(580+12*({\sqrt {(10*31.5)}}-{\sqrt {(10*31.5)}})/(667.5+1))$

Note that the 1 is replaced by zero for the away team in this calculation, and P(goal) is therefore 0.132385939 for the home team, and 0.131086142 for the away team. A difference of around 0.13%.

How it should work is that baseAttackCoeff is multiplied by the value of homeAdvantage in the relevant xml file (default is 4/3) for the home team:

$1-(580+12*({\sqrt {(10*31.5)}}-{\sqrt {(10*31.5)}})/(667.5*(4/3)))$

For the home team, the (4/3) would be 1 (no home advantage applied), so P(goal) is therefore 0.348314607 for the home team, and 0.131086142 for the away team – unchanged from the previous example – meaning if home advantage were applied correctly, the difference would be roughly 21.7%

Xkoranate RP bonus formulae

It is important to note how the bonus formula selected, as well as the RP effect entered, will affect the calculation of the skill values passed to the scorination algorithm.

Xkoranate has two methods of applying RP bonus when using the built-in "Bonuses" section, rather than directly applying RP bonus to rank or skill. Below is a summary of how these work, and their implications, as explained by Commerce Heights on the NS Sport forum

Before applying RP bonus, the participants’ skills are converted to a 0–1 scale:

$Skill={\frac {(NominalSkill-MinimumSkill)}{(MaximumSkill-MinimumSkill)}}$

Absolute

With absolute bonus, the RP-modified skill is calculated as:

$RPSkill=Skill+RPBonus\times \left\vert 1-{\frac {1}{(1-RPEffect)}}\right\vert$

The RP bonus effect dictates how much of the total RP-modified skill is accounted for by the RP bonus. For example, if the RP bonus effect is 20%, then the RP-modified skill is the participant’s skill plus 25% of its RP bonus, so that RP bonus accounts for 20% of the total. The maximum RP-modified skill is now 1.25, so all values are divided by 1.25 before they’re fed into the scorinator.

Relative

With relative bonus, the RP-modified skill is calculated in the same way as with relative bonus, but instead of dividing by the maximum RP-modified skill, values are divided by the largest value actually achieved by a participant in the event (a participant who has been placed into a group, to be precise). For example, if the best participant in the event has a skill of 0.9 and an RP bonus of 0.8, for a total of 1.1 (0.9 + 0.8 × 25%), then everyone in the event will have their RP-modified skill divided by 1.1. If no participant has an RP-modified skill greater than 1, then no rescaling occurs.

The intent of this system is to allow athletes with the highest skill values to achieve world-class results without being penalized if no one in the event has accrued a high RP bonus (e.g., in the early stages of an event).