{"id":1597,"date":"2024-05-03T17:42:14","date_gmt":"2024-05-03T17:42:14","guid":{"rendered":"https:\/\/egoanalytics.me\/?page_id=1597"},"modified":"2026-04-05T18:32:17","modified_gmt":"2026-04-05T18:32:17","slug":"model-evaluation","status":"publish","type":"page","link":"https:\/\/egoanalytics.me\/index.php\/model-evaluation\/","title":{"rendered":"Model Evaluation"},"content":{"rendered":"\n
\n
\n
\n

Table of Contents<\/strong><\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

–<\/p>\n\n\n\n

Explanation\/Background<\/a><\/p>\n\n\n\n

Coach Score Average<\/a><\/p>\n\n\n\n

Actual vs Predicted<\/a><\/p>\n\n\n\n

Heat Check<\/a> <\/p>\n\n\n\n

Gambling Performance<\/a><\/p>\n\n\n\n

The Problem With GM Evaluation<\/a><\/p>\n\n\n\n

–<\/p>\n\n\n\n

Explanation\/Background<\/strong><\/p>\n\n\n\n

\n
\n

While most people would understandably be interested in the gambling results of the project, the quality of the organizational scores (ORG Score, Coach Score, GM Score) is foundational, as it reflects the quality of the actual model. The initial inspiration for this project was to come up with a method to accurately evaluate organizations as a whole via the Coaching Staff and the Front Office performances in the same way that people like to evaluate the football players using analytics. Ideally, the organizational scores from this project would be efficient representations of what happens on- and off-the-field, but how would one know? How do you know how good a Head Coach is? And how much better is that one than another one? Admittedly, it all feels very subjective. But one would think that the people working for these organizations and making the decisions would (overall, over time) know best. So, I have turned to what happens to the Head Coaches after the season to see how the metrics generated from this project match with what actually happened. In this case, Organizational Actions for HCs that can happen are Coach of the Year<\/strong>, Left Team<\/strong> (not fired or retired – often phrased as “stepping away”), Retired<\/strong>, None <\/strong>(will return as HC for the same organization next season), Fired<\/strong>, and Fired Midseason<\/strong>. While the Coach Score represents the performance of the entire Coaching Staff, these actions only track actions relating to the Head Coach. However, changes to Offensive and Defensive Coordinators can be found in the Notes of the Actual versions of the Organizational Scores<\/a> section of the website. On this page, Head Coach is interchangeable with Coaching Staff unless a specific Head Coach is referenced.<\/p>\n\n\n\n

–<\/p>\n\n\n\n

Coach Score Average<\/strong><\/p>\n\n\n\n

Theoretically, the higher the Coach Score, the less likely that the Head Coach is going to get fired and vice versa. That may not always be the case, but overall, the distribution of scores should match the actions taken by the organization. Below is a chart of the average Coach Scores across the 6 organizational actions listed above.<\/p>\n\n\n\n

\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
\n