This competition compares preseason rankings by position to actual end of the season numbers from expert sites all over the web. The positions of quarterback, running back, and wide receiver contribute 2/7ths each to the overall score, and the tight end position has a weight of 1/7th. Each position is also ranked individually.   

Players will be excluded from the competition for reasons such as an injury that cause substantial time on the sideline or significant role change (e.g. demotion, trade) not known prior to the beginning of the season. Players who were little known and not on many preseason lists but had surprisingly good numbers are likewise not included. All sites will be evaluated on the SAME set of players.

Through the 2010 season we scored sites based on the total summed absolute value difference between the preseason rank versus the end of the season rank. This method, while easily understood, placed a far greater weight on the players ranked lower on the list. For example, Adrian Peterson might be ranked anywhere from 1st to 4th place. Thus the greatest difference any site would have compared to any other site would not be more than 3. Looking at a player further down most lists may show  rankings from, for example, a high of 5th place to a low of  25th place. This more lowly ranked player can have an impact difference up to 19 places whereas Adrian Peterson’s maximum difference was only 3 places.

We wanted to provide a method that would allow for players to be ranked an a more even basis. This would keep a few players from dominating the results.

This need has brought about a change in methodology beginning with the 2011 season. Two methods were joined 50/50 to produce the final results.

These methods have been dubbed the Percentile Method and the Rotisserie-Ranking Method. Both are good methods and could stand individually on there own merits but taken together provide a uniquely distinctive final ranking.

Rotisserie-Ranking Method

This Ranking Method ranks each expert site for EACH player as is done in a rotisserie-style fantasy league. The expert site that ranks a player closest to the actual rank receives a score of 1 for that player. The site that ranks the same player the next closest to actual will get a score of 2. Lower numbers are better. All of these individual player scores are then summed and the lowest scoring expert site is placed in the top spot for that position.

Percentile Method

The Percentile Method ranks expert sites for EACH player from 0% to 100% depending on where the expert site ranking fell relative to the range of all expert sites. This range is determined from the best ranking to the worst ranking. This is done individually for each player. Therefore the expert site with the best ranking for a specific player would receive a score of 100% for that player and, conversely, the worst ranking site would score 0%. Each expert site receives a percentage for each player depending on where in the range their ranking fell. This is done individually for all players by position and then summed. The best scoring expert site would have the highest summed percents.

The Percentile Method is the most exact method for scoring. However, if an entry from an expert site is significantly different from the other sites (usually a crazy ranking) then the range for this player becomes abnormally large. This causes the expert sites other than the outlier expert site to have different percentages (skewed from what would be considered “normal”) than they would have received sans the outlier expert site.

This is where the Rotisserie-Ranking Method helps with outliers. An outlier under the Rotisserie-Ranking Method will not yield a lengthening of the range for a particular player. The outlier expert site would score only the adjacent ordinal value rather than being given a gap in the percents. There would likewise be no adjustment to non-outlier expert sites. This helps to dampen any outlier issues.

To sum up the ranking process, the Percentile Method is the most exact whereas the Rotisserie-Ranking Method helps to deal with outliers, or the unnecessary lengthening of the range of possible ranks for a certain player. Ranking players individually helps to give more equal treatment to all the players in the survey.